Open-source Laya models claim 33ms calibrated decisions, 7.8x faster than Jev

What Laya is

A launch post by the project's author, currently on Hacker News's front page, introduces Laya: an Apache 2.0 family of non-autoregressive models that answer structured questions with calibrated probabilities rather than generating text. The weights ship in a single Hugging Face repository hub, convaiinnovations/laya, with a Python SDK, and are positioned as an open alternative to Jev, the closed decision model that TypeSafe AI launched in September 2026.

The backstory is told from one side. The author says they built the same core idea in March 2025 — using PPO over sequence representations to predict turn-by-turn conversion probabilities in sales conversations — and released it with a paper (arXiv:2503.23303), open weights and a dataset, then formalized the framework in a second paper (arXiv:2510.01237). In their account, TypeSafe AI then launched Jev with what they see as the same non-autoregressive decision concept, but without technical papers, open weights or training datasets. Those characterisations come from the author, not from TypeSafe AI.

Three decision primitives

Laya evaluates typed questions against an input state — raw text, an email, a ticket or a JSON document — in one forward pass. According to the post, it exposes three primitives: choice, which picks an option from a dictionary and returns the key, a distribution over options and a confidence score; score, which places the state on an ordinal rubric and returns the expected level plus a distribution over ranks; and noul, a boolean question returning a calibrated probability from 0.0 to 1.0. Because outputs are purely numbers and probabilities, the post argues, the model cannot hallucinate and malformed JSON is impossible.

The models are bidirectional encoders: a ModernBERT-large checkpoint for English and an mmBERT-base checkpoint covering more than 100 languages, with a third specialised checkpoint also in the hub. Rather than three separate repositories, the roughly 2.5 GB bundle uses subfolder downloads, so the SDK fetches only the roughly 808 MB English or 647 MB multilingual weights a user actually needs.

The confidence trap in multilingual routing

The most substantive finding in the post comes from a 51-language sweep on the MASSIVE benchmark, which has 20 options and a 0.050 random baseline. The English checkpoint collapses outside Latin script while its reported confidence stays high: Khmer scored 0.000 accuracy at 0.952 mean confidence, Armenian 0.050 at 0.885, Hebrew 0.060 at 0.964, and Bengali 0.080 at 0.945. Across all 51 languages, the author says, mean confidence never dropped below 0.885 — whether accuracy was 82 percent or zero.

The lesson the author draws is that confidence gating cannot protect a pipeline, because a model's own scores give no warning that it cannot read the input script. Laya therefore routes before the forward pass, using a pure-Python router that inspects Unicode scripts across 22 alphabets and analyses Latin stopword distributions. Detection costs 0.09 to 0.73 ms — under 2 percent of the roughly 33 ms forward pass — and a preload option keeps checkpoints resident in memory to avoid a 7 to 10 second cold-swap penalty when traffic alternates between languages.

Benchmark claims against Jev

The post publishes a head-to-head table, with Laya's numbers self-measured and Jev's drawn from third-party studies (AbdelStark and nibzard) plus TypeSafe AI's own figures. On a 2,000-decision typed benchmark, Laya scores 0.766 against Jev's 0.727, which the author notes beats a 0.735 teacher ceiling. On AG News it claims 0.950 versus 0.910, and on DAIR Emotion 0.595 versus 0.480. Expected calibration error is 0.081 versus 0.246, roughly three times better. Median single-question latency is 32.8 ms versus a reported 236 to 276 ms for Jev 1.13.0, and batched latency is measured in tens of milliseconds against roughly 1.5 seconds.

The post is not fully internally consistent on Jev's speed: it earlier describes Jev's typical response time as around 150 ms, while the benchmark table lists a P50 of 236 to 276 ms. It also contrasts Jev's pricing of $0.042 per million input tokens with Laya's self-hosted, zero-subscription model. None of these comparisons have independent verification.

Why it matters

The argument underneath Laya is that the industry burns generative LLMs on reflex decisions — routing support tickets, flagging spam or prompt injection, grading urgency on a rubric — paying 500 to 2,000 ms of latency and real inference costs for answers that reduce to a single label or probability. Non-autoregressive, calibrated decision heads are one plausible fix, and an Apache 2.0 release makes the architecture reproducible and auditable in a way closed APIs are not. The confidence-trap finding also stands on its own: a model reporting 95 percent confidence while scoring zero is a direct caution for anyone relying on LLM-stated confidence in production. The obvious caveat is that this is a single-source story written by the project's author, with self-run benchmarks and unverified claims about a competitor, so the numbers should be treated as claims until independently reproduced.