The prompt you review is not the prompt you ship, dev.to post on context assembly warns

A dev.to post argues that the prompt teams review in version control is only a fragment of what actually reaches a large language model in production, and that the gap between the two is a recurring source of unexplained quality regressions.

What ships is assembled, not authored

In the post's framing, a typical pull request contains a 30-line change to a prompt template, while the corresponding production request carries around 40,000 tokens assembled at request time from six sources — four of which changed that week and none of which appear in the pull request. The template lives in git and gets reviewed; the code paths that assemble context are not treated as release artifacts.

Author Dima Kruglov lists what typically goes into a request for a RAG-plus-tools system: the system template, top-k retrieved chunks from an index that is rebuilt on its own schedule, tool JSON schemas whose descriptions are effectively prompt text written by whoever last edited the tool, a conversation summary produced by another model call, a user-profile or tenant configuration blob, and few-shot examples chosen at runtime by a selector. Each is a dependency with its own version and deploy cadence. When someone reindexes, or a backend engineer renames a tool parameter, no prompt review is triggered. The consequence, Kruglov writes, is that when quality drops on a Tuesday and no code shipped, there is no diff to examine.

Hash what the model actually sees

The central recommendation is to treat the assembled prompt as a build output. Compute a content hash over the inputs that can change model behaviour — model id and version, template version, the retrieval snapshot id for the index build rather than the query, the fully rendered prompt, and the decoding parameters that matter — and log it with every trace.

Fingerprints should always be stored, while full rendered prompts should be kept only sampled or redacted, with the choice of how much full text to retain governed by compliance rules rather than debugging needs. Kruglov also warns against hashing the wrong things: a fingerprint that includes a timestamp, a request id or a per-user greeting has a near-zero collision rate and zero diagnostic value. He recalls a team that spent a day chasing a prompt hash that changed on every request, only to find the culprit was a locale string.

An assembler with trust levels and budgets

No framework is needed, the post argues — an assembler requires three properties. Every section carries a trust level, every section carries an eviction priority, and a pinned section that exceeds the token budget raises a hard error instead of being silently trimmed.

An accompanying Python sketch marks retrieval text, tool output and user uploads as untrusted and passes them through a regex screen that filters instruction-like phrases such as "ignore previous instructions", which Kruglov characterises as a rough heuristic rather than a guarantee, with hits logged and routed to a stricter path. When the budget is tight, lower-priority sections are dropped first. Two design choices stand out: sections render in their original insertion order, because reordering identical content changes model behaviour and makes evaluation deltas unattributable, and the pinned flag converts budget overruns into an exception rather than degraded output.

Why it matters

The post describes a failure mode that a standard LLMOps stack — evals, tracing, a gateway — does not catch, because the artifact that gets reviewed and the artifact that gets shipped are different objects. Fingerprinting the fully assembled request makes "quality dropped but nothing shipped" answerable with a single query during an incident. It also reframes governance: retrieval snapshots, tool schemas and runtime selectors become versioned dependencies of the prompt, and concerns like injection screening and context-budget enforcement move into the assembly layer, where they can be enforced deterministically instead of discovered after the fact.