Privatemode turns GLM-5.3-Flash into a single-pass decision model with per-option probabilities

A Privatemode blog post, currently on Hacker News' front page, shows that an unmodified LLM can act as a fast decision engine: given some state and a fixed set of options, it returns one choice with a probability for every option — all in a single forward pass, with no fine-tuning. In the post, dated September 24, 2026, author Marko Rosenmüller and his team evaluated the technique with GLM-5.3-Flash served on vLLM and report results on par with TypeSafe's Jev, a purpose-built decision model, on decision accuracy and speed. It also adds typed decisions on images, which the post says Jev cannot handle.

The problem with prompting for decisions

Much of what software asks an LLM is a multiple-choice question in disguise: which team gets this ticket, or whether a contract clause belongs in the liability section. The standard fix is to demand a JSON response drawn from a fixed set of values. That works, but the model must generate a whole object — and a reasoning model may think for hundreds of tokens first — while offering no confidence signal unless you explicitly request one. Per the post, speed, cost and missing confidence have kept general-purpose LLMs out of high-volume decision pipelines. Specialized "System One" models such as Jev and Convai's Laya exist for exactly this job: they take state plus named options and return a chosen option with a confidence value for each.

Reading the decision off the logits

The technique exploits how LLMs work under the hood: a model never writes text directly, it emits a probability distribution over its vocabulary at each step, and generation repeats that process. If the only output you need is an index, one step is enough.

The prompt contains the state, the question and the numbered options as JSON, and asks the model to answer with choice_index: followed by an index. The assistant turn is prefilled so the prompt already ends with choice_index:, which makes the next token one of the option indexes. Rather than reading the token the model would emit, the implementation reads the probability it assigned to every option index at that single position, restricts the vocabulary to those indexes, renormalizes, and picks the most probable. The result is a full distribution over options — doubling as a confidence measure for routing uncertain cases to a human — for the cost of one forward pass.

Making it work on vLLM

According to the post, the implementation uses the /chat/completions endpoint with continue_final_message and add_generation_prompt: false, so the model continues the prefilled turn instead of starting a new one; the same call accepts images alongside text, which is what enables image-based decisions. Three vLLM details mattered:

• allowed_token_ids confines output to valid option indexes by masking all other tokens; the authors set it but call it a guardrail rather than a requirement.
• top_logprobs is insufficient because it reports the distribution before restriction, letting formatting tokens such as a leading space occupy the top slots and pushing some options off the list as apparent zeros. vLLM's logprob_token_ids returns log probabilities for exactly the token ids requested.
• Index token ids depend on the tokenizer, and digits are not always single tokens — GLM-5.3-Flash has one token for "12". Rather than shipping model-specific tokenizers, the library asks the serving endpoint to echo a prompt and reads the exact tokenization from the reply.

The Python library and benchmark harness are published openly as edgelesssys/privatemode-decisions and edgelesssys/privatemode-decisions-benchmark, and the authors state every number in the post can be recomputed from the benchmark.

Benchmark setup and headline results

The comparison used 29 public, labeled datasets with 2 to 151 options each, covering intent routing, sentiment, topic classification, moderation, entailment, question answering, legal text and scanned documents, in English and German. Three systems received identical state, option lists in the same order and the same instruction: GLM-5.3-Flash with this technique on Privatemode, Jev, and Laya. Jev and Laya ran at default settings, and the GLM prompt was not tuned on these datasets.

Each dataset ran twice. Even at temperature 0, the GLM setup and Jev changed up to 3.5% of answers between identical runs, because batching and floating-point arithmetic keep a forward pass on a busy server from being bit-reproducible; smaller differences were treated as noise. Within those limits, the post reports parity with Jev on accuracy and speed, with playground answers typically returning within a few hundred milliseconds. The model solves most classic trick questions, though not all of them.

Why it matters

If an off-the-shelf model matches a purpose-trained decision model, the barrier to putting LLMs into high-throughput decisioning largely disappears: each decision costs roughly one token of compute instead of a full JSON generation, and the answer ships with a probability distribution — exactly what confidence thresholds, abstention and human-in-the-loop routing need. Image support comes free from the underlying chat endpoint, and the approach is model-agnostic in principle since token ids are discovered from the server rather than hard-coded. The caveats are ones the authors flag themselves: a custom benchmark, competitors at defaults, and up to 3.5% run-to-run variance, so small accuracy deltas should not be over-read. And because nothing is trained in, calibration quality is inherited from the base model — worth validating before these probabilities drive production thresholds.