deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Redis 8.10 compact hashes: real memory savings need HIMPORT or a restart

Hands-on benchmarks of Redis 8.10's compact hashes show memory savings swing from 20% to 68% depending on schema shape, and hashes written with plain HSET only convert after a restart.

Redis 8.10 compact hashes: real memory savings need HIMPORT or a restart

Independent benchmarks of Redis 8.10's compact hashes show the feature's advertised memory savings swing widely with schema shape — from roughly 20% to nearly 68% in the tests — and that hashes written with ordinary HSET commands are never converted until an operator either switches to the new HIMPORT command or enables new RDB-load settings and restarts the server. The tests were published on dev.to and ran Redis 8.10.1 in a Docker container.

Compact hashes shipped with Redis 8.10 in July: when many hashes share the same field names, Redis can keep a single copy of those names instead of repeating them per key. According to the dev.to write-up, the release blog claims up to 50% lower memory and 2x higher loading throughput.

Savings ranged from 20% to 68%

The author loaded 100,000 hashes with a four-field user-profile schema (name, email, country, last_login) two ways: plain per-key HSET, and HIMPORT, a new command pair that declares a field list once per connection with HIMPORT PREPARE and then loads values against it. Plain HSET used 159.9 bytes per hash; HIMPORT used 127.6, a 20.2% reduction and well below the advertised ceiling. A second schema built to favour the feature — eight fields with long names such as account_status_code and two_factor_enabled_flag holding short values — went from 271.3 bytes per hash to 87.3, a 67.8% reduction. The savings come from the field names themselves, so the more a schema's memory is names rather than values, the bigger the payoff.

Throughput improved 35%, not twofold

Loading 500,000 hashes three times per method, the fastest plain-HSET run took 2.698 seconds (185,323 hashes per second) and the fastest HIMPORT run 1.992 seconds (251,004 per second): a 35% improvement, not the claimed doubling. The author speculates the blog's figure may reflect wider schemas or network-bound clients, where omitting repeated field names on the wire matters more than it does over a local container pipe.

Existing HSET data converts only via HIMPORT or restart

This is the operational catch. After 100,000 same-schema hashes were loaded with HSET, INFO stats showed hash_templates at zero: Redis leaves live hashes alone rather than detecting that they share a field set and consolidating them. Conversion happens only through HIMPORT, or when an RDB file reloads with three new configuration parameters set — hash-rdb-load-min-template-entries, hash-rdb-load-max-template-entries and hash-rdb-load-template-disassembly-threshold — all of which default to zero, meaning off. With the settings enabled, a BGSAVE followed by a container restart converted all 100,000 hashes into one shared template, dropping used_memory from 17,459,464 bytes to 14,182,656, an 18.8% cut consistent with the direct comparison. The practical consequence, per the author: a primary that stays up indefinitely and never receives HIMPORT writes gains nothing from the feature.

Small groups and stray fields can cost memory

Templates carry fixed overhead. MEMORY USAGE reported a key in a two-key template at 156 bytes and a three-key template at 129, both worse than the 112-byte plain-hash baseline; the crossover sits around four to five keys, and savings plateau near 100 keys at 76 bytes.

Mutating a templated hash also behaves unexpectedly. Adding a field outside the schema with HSET did not revert the key to plain encoding — Redis forked a new single-key template for it, pushing the key from 76 bytes to 265, while an HDEL on another key produced 208 bytes. Both exceed the plain baseline; the untouched keys were unaffected.

HIMPORT fieldsets are scoped per connection

Fieldsets prepared with HIMPORT PREPARE live on the connection, not the server. Wrong value counts and unknown fieldset names return errors, and a fieldset prepared in one redis-cli session cannot be used from another — which means connection-pooled clients must run PREPARE on every pooled connection or pin an import to a single connection. RESET drops fieldsets immediately. Concurrency itself held up: ten parallel redis-cli --pipe processes, each preparing its own schema and writing 2,000 disjoint keys, finished with zero errors in under a third of a second combined.

Why it matters

For operators planning an upgrade, these benchmarks frame compact hashes as an opt-in encoding with conditional wins rather than an automatic improvement. Existing HSET workloads need either a rewritten write path or new configuration plus a restart or failover on every node holding the data. Schemas dominated by long field names and short values gain the most; small groups of hashes under one fieldset, or applications that add stray fields, can end up using more memory than before. Measuring your own keys with MEMORY USAGE before committing is the reliable way to tell which side you fall on.

  • #redis
  • #memory-optimization
  • #benchmark
  • #databases
  • #performance

Related posts