Open-source Jev-like model scores text options in one pass, echoing TypeSafe's Jev

A project named jevlike, published on GitHub by vinnylarouge and surfaced on the Hacker News front page, is an independent attempt to rebuild the interface of TypeSafe's commercial Jev model. According to the repository, Jev accepts a piece of text and a list of N text options and emits one probability per option in a single pass, rather than generating an answer token by token. TypeSafe has not published Jev's design; the project instead offers an independently trained starter model with the same input and output shape.

How the architecture works

Each option becomes a query vector, a compact numeric embedding of its text. The query computes attention weights over the tokens of the context text, and those weights produce one context vector per option. A shared dot product turns each option-and-context pair into a single score, and a softmax across the options converts the scores into a probability distribution.

Two encoder paths are supported. The default learns byte embeddings from scratch, which is cheap but truncates context to 192 bytes and each option to 32 bytes. The optional path plugs in any compatible Hugging Face encoder, kept frozen while only a small scorer head trains; the README names Qwen/Qwen2.5-0.5B as an example. Checkpoints hold the head and the encoder name, not the encoder weights, so reloading requires access to the same Hugging Face model.

Training and evaluation

Training data is one JSON object per line containing a context, an options list and the zero-based index of the correct answer. Rows can carry different numbers of options, with a minimum of two. Command-line tools generate synthetic data, train, evaluate and score new menus, and training runs on CPU, Apple MPS or CUDA.

Evaluation prints top-1 and top-3 accuracy plus expected calibration error, which compares model confidence with observed accuracy. It also reports a shuffled-context control that pairs each menu with the wrong context; a useful model should beat that baseline, the README says. For real datasets, the project advises keeping all records for one customer or page in a single split so near-duplicates cannot leak into the test set, and evaluating once on a held-out file. A Wikispeedia script downloads the public SNAP archives and builds next-click prediction data, citing Robert West and Jure Leskovec's WWW 2012 paper on human wayfinding.

The reported numbers

In the experiments behind the project, the one-pass scorer reached about 98% accuracy on synthetic menus. On target-disjoint Wikispeedia next-click data, a frozen Qwen2.5-0.5B encoder plus the scorer reached 26%, against roughly 8% for shuffled and random-encoder controls, while a small model trained from scratch on 40,000 clicks reached 29%. At eight options, one pass was roughly 100 times faster than a small decoder forced to write 400 tokens.

The repository is explicit that these figures describe local experiments, that the speed comparison used a small local decoder rather than a large commercial model, and that it neither matched Jev's quality nor reproduced TypeSafe's private training method.

The same head plays games

The option-attention head can also score controller buttons from image patches. A ten-second demo film joins two five-second windows: Doom combat on seven buttons and a chess controller using five keys. The Doom window, from the joint checkpoint, averaged 0.60 kills and -97.50 reward across its ten recorded episodes. The chess window, from a chess-only checkpoint, scored 4 wins, 46 draws and 0 losses in 50 sampled games against a random mover, but 0 wins, 2 draws and 48 losses against Stockfish level 0. The repository cautions that the windows were chosen for activity, not as typical-play or competence claims. Both games import the visual scorer from one shared module, and the release includes the single-game checkpoints plus a shared 12-option checkpoint.

Stated limitations

The project lists further limits: it is a research starter, not a Jev copy; accuracy depends on data quality, split quality and the encoder; the byte encoder is weak on language meaning; the pretrained path may download a large model and needs more memory; and one-pass scoring requires the complete option list before prediction. The code is MIT-licensed, with downloaded datasets and pretrained models keeping their own terms.

Why it matters

Most work with large language models runs through autoregressive generation, which is wasteful whenever the task is simply choosing one item from a known list, such as routing a support ticket, picking a next link or selecting a menu entry. This project frames an alternative: score every option directly in one pass, read off calibrated probabilities, and cut latency and compute accordingly. It is also a case study in openly reverse-engineering a commercial model's interface, giving developers a runnable, MIT-licensed base for testing whether the single-pass paradigm holds up on their own data.