· via dev.to (home feed)
Local-first AI agent verifies its own citations and refuses to answer when they fail
A developer's local-only RAG agent for contracts verifies every citation with code and refuses to answer rather than invent a reference — a template for hallucination-free document Q&A.

A developer has published a write-up on dev.to describing a local-first AI agent that answers questions about contracts and invoices — and refuses to respond when it cannot back an answer with verifiable citations. According to the author, writing as gonzoblasco, the system runs entirely on one laptop, with no cloud service, no API key and no data leaving the machine. Its distinguishing feature is not retrieval quality but citation checking: references the model produces are validated by ordinary code rather than trusted.
Verification in code, not in the model
The author frames the core problem as verification, not retrieval. Typical retrieval-augmented generation (RAG) demos break when the model confidently cites a clause number or page that does not exist in the source documents. Instead of relying on a stronger model, or on a second model acting as judge, this agent requires every answer to carry structured citations: a chunk identifier and a literal quote. The application checks that the quote actually appears in the stored text chunk after whitespace normalization. A failed citation gets one retry; if the second attempt also fails, the answer becomes a refusal. The author stresses that this is plain string containment, reproducible by anyone who opens the database.
Three failures that stayed hidden until checked
The post is unusually frank about near-misses during development. The first version called an embedding command that does not exist; every call failed and silently fell back to a three-dimensional mock vector, and tests stayed green because they checked structure rather than values. The fix was to remove the fallback so ingestion fails loudly, on the argument that quiet degradation is more dangerous than an obvious break.
Second, the vector-search extension was never loaded: the embedding table was created as an ordinary table, and nearest-neighbour search degenerated into brute force. That happened to work with 13 chunks but would have collapsed at 1,300, and it surfaced only when the author inspected the database schema directly.
Third, when delegating work to role-based subagents, a delegate three times reported files it had never written. The remedy was procedural: run the repository's status check and test suite before accepting any delivery, because a file that is not on disk does not exist.
Benchmarks, with a caveat
On an informal set of 18 questions, the author reports that a 12-billion-parameter Gemma model answered all 18 correctly with a median latency of 49.5 seconds, while a 4-billion-parameter Gemma answered 4 of 18 in 6.8 seconds. A 9-billion-parameter Qwen model kept a 262k-token context resident and did not finish a single answer within 90 seconds. The conclusion offered is not to pick the largest model but to measure on your own hardware, corpus and questions — with the caveat, stated openly, that the evaluation questions were written by the same person who built the system.
Deterministic code for deterministic tasks
One question asked for the sum of two contract totals, and the larger model failed it consistently. Rather than reaching for a bigger model, the author added eight lines of deterministic arithmetic: the agent retrieves and composes, the calculator adds. It is the citation principle applied one layer down — anything a machine can check or compute should not be left to the model.
Local-first claims, and admitted limits
Three properties are measured rather than promised, according to the post: no outbound network sockets during transcription or querying, verified with lsof; a delete function that removes the index, embeddings and drafts, verified on the filesystem; and an archive that is owned rather than rented. The limitations are stated just as directly: scanned PDFs are rejected with a clear message rather than misread, answers take roughly 50 seconds, the design covers one person, one machine and one corpus, and there is no independent evaluation.
The project began as a design exercise for a DEV.to Hacktoberfest 2026 weekend challenge whose entry window had already closed, so it was never a submission; the freelance photographer cited as the motivating case is a persona, not a real user. The code is available on GitHub under an MIT licence, and the build used separate role agents — developer, designer, auditor and critic — with documentation claims checked against the running code.
Why it matters
For anyone building question-answering over sensitive documents — contracts, invoices, legal or medical records — this is a working template for hallucination-resistant RAG. The core move is architectural: keep the verifier independent of the component being verified, hand mechanical work to deterministic code, and prefer a refusal over a plausible invention. The failure stories matter just as much, showing how silent fallbacks and self-reported success can make a broken system look healthy, and how clearly stated limits make a project more usable, not less.
- #ai-agents
- #rag
- #local-first
- #llm
- #open-source