Valkey 9.1 hash field TTL triples memory for the per-user token pattern in its own docs

A benchmark published on dev.to reports that Valkey 9.1's hash field TTL feature roughly triples memory use for the exact pattern Valkey's own documentation uses to demonstrate it. Testing Valkey 9.1.2 in Docker with 20,000 keys and measuring via INFO memory, the author found the hash-based pattern consumed about 5.2 MB, versus roughly 1.7 MB for the equivalent plain string keys.

What was measured

Valkey 9.0 introduced TTLs on individual hash fields rather than whole keys, and 9.1 added HSETEX, which sets a field and its expiry in one round trip. Valkey's blog illustrates it with a per-user auth token: HSETEX user:123 EX 900 FIELDS 1 auth_token, followed by the token value. Read as a drop-in replacement for a plain string key with SET and EX, it looks like a tidy upgrade.

The dev.to author, writing under the handle alexgeorgiev17, compared three layouts for 20,000 expiring items, driving the server with redis-py 8.1.0 over the container's exposed port. A hash per item with a field TTL set through HSETEX used 5,226,560 bytes, or about 261 bytes per key, in hashtable encoding. A hash per item with no TTL used 1,754,720 bytes, about 88 bytes per key, in listpack encoding. A plain string key plus EXPIRE used 1,733,680 bytes, about 87 bytes per key. In other words, the documented hash pattern costs three times the memory of the string pattern it was positioned against, and it carries the hash's own bookkeeping on top.

The cause is an encoding switch

Using OBJECT ENCODING, the author confirmed that setting even one field TTL flips a hash from listpack to hashtable, no matter how few fields it holds. The default hash-max-listpack-entries value of 512 made no difference. The encoding conversion, not the expiry tracking itself, accounts for nearly all of the extra bytes. According to the post, this is a known gap: Valkey issue #2618 describes the behavior and proposes storing expiry data inside the listpack for small hashes, but as of 9.1.2 that change has not landed.

The layout where the feature pays off

The post also tested the shape the feature appears designed for: a single hash holding 20,000 fields, each carrying its own TTL. Both the TTL and no-TTL variants were hashtable-encoded, since 20,000 fields far exceeds the listpack ceiling. The difference came to 23.44 bytes per field, which the author notes lines up with the 16-to-29-byte range Valkey maintainers cite in issue #2618. The takeaway offered is that the cost is about encoding, not about TTL tracking: once a hash is large enough to live in a hashtable anyway, per-field TTLs are close to free, while a single TTL on a small hash forfeits listpack entirely.

Other findings from the tests

• Reads showed no penalty. HGET throughput against a 5,000-field hash was statistically indistinguishable with and without TTLs on every field, around 4,885 to 4,889 ops per second at roughly 205 microseconds per call, though the author cautions that latency was dominated by Docker's port-mapped network round trip.
• Concurrency changed nothing. Ten pipelined threads achieved overlapping throughput figures across per-thread hashes, one shared hash, and independent string keys, roughly 62,000 to 69,000 ops per second, because Valkey executes commands on a single thread regardless of client count.
• Field TTLs survived a full container restart after a background save and kept counting down rather than resetting.
• Expiry worked on schedule: 2,000 fields given two-second TTLs all disappeared within a single 200-millisecond polling window, with INFO stats reporting expired_fields at 2000 and expire_cycle_cpu_milliseconds at 3. There is no expired_subkeys stat.
• Error paths worth knowing: HEXPIRE returns WRONGTYPE against string keys, rejects negative expire times and incompatible NX, XX, GT and LT combinations, and returns -2 for missing fields or hashes. HEXPIRE with GT on a field with no TTL is a documented no-op returning 0.

Why it matters

Field TTLs are a genuinely useful capability, but the published example invites operators into the one layout where they are most expensive. Anyone who adopted the per-user token pattern from Valkey's HSETEX documentation may be paying roughly three times the memory of their previous string-key design, silently, with no error and no read-path slowdown to signal it. Until issue #2618's listpack support lands, teams should size their hashes deliberately: pack many expiring fields into shared, hashtable-scale hashes, keep single expiring values as plain string keys, and monitor OBJECT ENCODING alongside the expired_fields stat to catch hashes that quietly lost their compact representation.