· via Hacker News – Front Page (native)
Zerodha open-sources Blobasaur, a sharded blob store that runs on SQLite
Zerodha replaced its aging SSDB setup with Blobasaur, an open-source Rust server that shards SQLite across commodity machines and speaks the Redis protocol, serving thousands of blob reads and writes per second.

A blob store built on SQLite
Indian brokerage Zerodha has open-sourced Blobasaur, a sharded blob store written in Rust that uses SQLite as its storage engine, according to the company's engineering blog. The server speaks the Redis protocol, ships as a single static binary, and is already running in production, where it has fully replaced the system that previously handled the workload.
That workload comes from Kite, Zerodha's trading platform, which serves roughly 20 million customers who heavily customise their screens: multi-chart layouts, indicator templates, vector drawings, and workspaces assembled from drag-and-drop widgets in a Terminal mode. Backend services store these preferences as opaque, mostly JSON blobs ranging from a few kilobytes to several megabytes. Zerodha says the store sustains between thousands and tens of thousands of blob operations per second — traffic that is bursty and hard to predict because it tracks stock-market activity — while keeping latency in the single-digit milliseconds on commodity hardware.
From CouchDB to SSDB to a rewrite
The blobs originally lived in CouchDB, which struggled to keep up as throughput grew. Zerodha then moved to SSDB, a disk-backed store written in C++ that happens to speak the Redis protocol. SSDB ran reliably in production for years, but two problems forced a replacement: the project became unmaintained, and because C++ sits outside the team's expertise, forking it was not realistic. SSDB is also a single-node system, and its limits at one point forced a 1 MB cap on blob sizes because latency suffered on larger payloads.
The requirements for a successor were a simple key-value API, multi-megabyte blob support at high throughput, horizontal scalability, compression, and low operational cost.
Why the usual suspects fell short
Zerodha evaluated a long list of candidates, and the blog post explains why each was rejected. ScyllaDB, already in the company's stack, showed significantly lower throughput and higher resource use on large blobs, plus tombstone tuning headaches from frequent overwrites and TTL expiry. PostgreSQL, where blobs were tested as BYTEA columns, performed poorly on both speed and disk efficiency. YugabyteDB was judged too large a system and not focused on blob storage, while RocksDB had the right primitives but was not built for high-throughput multi-megabyte writes. Ceph's operational complexity meant the team never got as far as testing it.
SeaweedFS came closest, but the workload is overwrite-heavy and SeaweedFS appends new versions before compacting, which can grow disk usage uncontrollably; it also lacked per-blob TTLs and required a separate database to map blob IDs to user keys. S3 and MinIO carried per-request latency and cost problems at thousands of operations per second — writes make up half the traffic — along with coarse lifecycle-based expiry. A raw filesystem approach, one file per blob, would have meant building atomic writes, batching, TTL indexes, and metadata management from scratch.
Why SQLite and Rust
Two things pushed the team toward SQLite: experiments with it for another project, and SQLite's own claim that it can be 35% faster than direct filesystem I/O. Raw files do pull ahead of SQLite above roughly 100 KiB, but Zerodha compresses blobs before writing them, and 99% of compressed values land below that threshold. SQLite also brought ACID transactions, crash recovery, and standard tooling — any shard on disk can be inspected with the sqlite3 CLI or queried with plain SQL.
Durability is deliberately relaxed: the store holds UI preferences, not financial records, so a lost write after a power failure costs a user a chart tweak, not a trade.
Rust was chosen over Go because Go's SQLite bindings depend on CGo, which complicates static builds, pins OS threads during C calls, and adds garbage-collection pressure when allocating and compressing multi-megabyte buffers. Using Rust crates such as rusqlite and tokio, Blobasaur calls SQLite's C API directly with predictable memory use.
Sharding around SQLite's write model
Redis protocol compatibility was a hard requirement, since existing Go and Python services already used standard Redis commands like GET, SET, HGET, HSET, DEL and EXPIRE; a drop-in replacement meant no client-side changes. On the storage side, SQLite in WAL mode allows many concurrent readers but only one writer at a time per database — a real constraint given that writes are about half of Zerodha's traffic. Blobasaur therefore partitions data into shards spread across multiple machines, so no single node or database file becomes the bottleneck.
Why it matters
Blobasaur is a strong data point in SQLite's quiet rehabilitation as a server-grade engine. For a well-bounded workload — key-value blobs, simple CRUD with TTLs, relaxed durability — a sharded arrangement of embedded databases on cheap hardware beat purpose-built distributed stores on cost, simplicity and speed. The Redis wire protocol makes adoption nearly free for existing services. And the design is a useful case study in right-sizing: Zerodha did not need a consensus-based distributed database, it needed fast, disposable storage, and matching the tool to that reality produced a single-binary system one team can operate.
- #sqlite
- #rust
- #open-source
- #blob-storage
- #distributed-systems