deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

MongoDB 8.2 encrypted substring search costs 20x more per insert than encrypted equality

A dev.to benchmark finds MongoDB 8.2's preview encrypted substring search costs about 20x more per insert and 79x the storage of encrypted equality, with hard limits on combining query types.

MongoDB 8.2 encrypted substring search costs 20x more per insert than encrypted equality

Substring queries reach encrypted fields

MongoDB 8.2 shipped public-preview support for prefix, suffix and substring lookups against Queryable Encryption fields, delivered through three new aggregation operators: $encStrStartsWith, $encStrEndsWith and $encStrContains. Before this release, an encrypted field could only be queried by equality or range — there was no encrypted equivalent of a SQL LIKE '%foo%'. A benchmark published on dev.to measures what the new capability actually costs, and the headline number is steep: inserting documents into a collection with a substring-searchable encrypted field ran roughly 20 times slower per document than the same field indexed for encrypted equality.

The test setup, as described in the post, ran MongoDB 8.2.12 Community Edition as a single-node replica set — a requirement for Queryable Encryption — with PyMongo 4.18.1, the crypt_shared query-analysis library, and a local KMS key. The author also corrects a common assumption: automatic Queryable Encryption does not need the Enterprise build. crypt_shared is a free download, and the whole configuration ran on stock Community Edition without complaint.

The write and storage bill

Inserting 200 documents took 4.6 seconds into a collection with one substring-searchable encrypted email field, versus 232 milliseconds with equality-only encryption and 7 milliseconds unencrypted. Per document, that works out to 22.9 ms for substring, 1.16 ms for equality and 0.03 ms for plaintext, with average document sizes of roughly 24 KB, 489 bytes and 133 bytes respectively.

Storage is the sharper penalty. Counting the hidden ESC and ECOC metadata collections that back the encrypted index, 500 plaintext documents occupied 66.5 KB, equality encryption 319.5 KB, and substring encryption 25.3 MB — around 380 times plaintext and 79 times equality. The dev.to post explains why: with the query length range set to 3–10 characters, MongoDB has to pre-generate a matchable token for every substring in that range, and the ECOC collection held about 173 tokens per email address — 86,500 entries for 500 documents.

Hard limits on configuration

The bounds are enforced, not advisory. According to the post, inserting a 47-character string into a field capped at a maximum length of 40 fails on the client before anything reaches the server, as does querying with a two-character substring against a configured minimum of three. A plain $regex against an encrypted field is rejected outright rather than silently returning nothing.

The bigger design constraint is that a field can carry at most two query types, and the only legal pair is prefixPreview plus suffixPreview. SubstringPreview cannot be combined with anything — not even equality — so a schema needing both substring and equality matching on the same logical value currently requires two separate encrypted fields holding the same plaintext. A combined prefix-and-suffix field did work correctly in testing, at 4.3 ms per insert (still about 130 times the plaintext baseline) and roughly 2,500 bytes per document, a tenth of what substring alone needed.

Query speed and a benchmarking pitfall

Against 500 documents, $encStrContains matched 112 documents for a test string in a fastest run of 25.3 ms, while the equivalent plaintext regex took 1.0 ms — about 25 times slower, with identical result sets, so only cost was in question, not correctness.

The author's first concurrency test suggested a 17x slowdown with ten Python threads, but that turned out to measure the interpreter lock rather than the database: query analysis for these operators happens client-side inside crypt_shared and is CPU-bound, so threads serialize regardless of server behaviour. Rerun with ten separate processes, each with its own client, the slowdown was closer to 2x. It is a useful caution for anyone benchmarking client-side crypto from Python.

For monitoring, db.serverStatus().fle exposes live counts of which query types are in use across encrypted fields on a deployment — the one at-a-glance adoption check the author found. MongoDB's own documentation, cited in the post, marks the feature as preview: unsuitable for production, and incompatible with the eventual general-availability version.

Why it matters

These are the first hard numbers most teams will see for a feature that is trivial to enable but expensive to run. Anyone weighing Queryable Encryption's substring search now has concrete write, storage and latency costs to model: roughly 20 times the insert cost of equality encryption, storage two orders of magnitude beyond plaintext, and matching about 25 times slower even on a tiny dataset. The benchmark also settles a licensing question — the stack runs on free Community Edition — and documents schema constraints, such as substring's inability to share a field with equality, that force design decisions today even though the feature is preview-only.

  • #mongodb
  • #encryption
  • #database
  • #benchmark
  • #nosql