deniz.in

Markets

Weather

Loading weather

· via Hacker News – Front Page (native)

Ampbase runs on object storage alone, hand-rolling the database features it lacks

A guest post on the Tigris blog explains how Ampbase persists its entire control plane to object storage, implementing unique constraints, transactions, indices and history tables on top of conditional writes.

Ampbase runs on object storage alone, hand-rolling the database features it lacks

A control plane with no database underneath

A guest post on the Tigris Data blog, currently on the Hacker News front page, explains how Ampbase runs its control plane entirely on object storage. Written by Ampbase founder JP, it follows up on an earlier article about the database engines the company chose not to use: there is no relational database in the stack, only Tigris — an object storage service that is itself built on FoundationDB, a distributed key-value store. The author is upfront about the genre, joking that posts declaring a database unnecessary tend to age badly and often precede a sheepish migration to Postgres.

Two layers of buckets

Everything lives in two tiers. A single global directory bucket holds the list of organizations, with keys for each org's metadata, members (keyed by the SHA-256 hash of the email address, serving as the index), billing state (the compare-and-swap target), channel metadata, API tokens, audit events and an operations queue. Each customer then gets a bucket of its own containing channel slugs (the uniqueness constraint), configuration and bundle metadata plus their versioned records acting as a history table, an active-config pointer that is overwritten in place, and an event stream.

Records started life as JSON but moved to Protocol Buffers once marshaling costs grew beyond expectations; the storage layer handles both formats, so older JSON records still load. According to the post, Protobuf eliminates whole categories of database work — migrations, connection handling, schema management — at the price of field names being effectively permanent, a pain the author compares to renaming columns in Postgres, MySQL or SQLite.

Isolation was pushed down into the infrastructure. Rather than one bucket with per-customer prefixes and complicated IAM policy, Ampbase uses Tigris's Partner Integration Program: a single call creates a Tigris organization, a bucket and access keys scoped to that customer. Because one customer's credentials cannot address another's data, there is no tenant filter to forget in application code — which the author identifies as the mistake platform builders most often make.

Building database guarantees from two primitives

The post breaks a database engine down into six things: strong read-after-write consistency, conditional writes, unique constraints, transactions, indices and history tables. Tigris supplies the first two off the shelf; Ampbase implemented the other four itself.

Strong consistency is relatively new to object storage. The post notes that object stores lacked a strong consistency model until around December 2020 and points to a writeup by Werner Vogels on the internals. Tigris's distinguishing property is that bucket data, not just bucket names, is global. Reads are strongly consistent within a region, but global replication makes cross-region behavior eventually consistent, so deciding where strong consistency was genuinely required versus where eventual consistency sufficed became a central design question.

Conditional writes amount to compare-and-swap. Tigris supports HTTP preconditions on writes: If-None-Match: * succeeds only if the key does not yet exist, while If-Match carrying an ETag succeeds only if the object is unchanged since it was read. Once compare-and-swap is available, the author contends, you hold the core primitive underlying every database engine.

On uniqueness, the post argues that unique indexes exist less for lookup speed than to prevent duplicates — the reason you index an email column is to stop storing it twice. Ampbase combines content-aware storage with conditional writes so identical data cannot be written twice. The published text cuts off mid-explanation, so the detailed mechanics of the transaction, index and history implementations continue beyond the excerpt, though the storage layout described above already signals how each primitive maps onto specific keys.

Why it matters

The article works as a dissection of what databases actually do. Unique constraints, transactions, indices and history tables are features most teams consume without thought; seeing them rebuilt from conditional writes and consistency guarantees clarifies what a database is really selling you, and when it is worth doing without one. It also shows the feasibility bar moving: object storage gained strong consistency only in late 2020, and conditional writes now sit in the same layer, making S3-style stores a plausible persistence substrate for control-plane-scale workloads. The isolation design is a transferable lesson in its own right — scoping credentials per customer at the infrastructure level removes a whole class of multi-tenant bugs that application-level filtering invites.

  • #object-storage
  • #databases
  • #architecture
  • #distributed-systems
  • #tigris

Related posts