Security researcher rebuilds LLM agent memory as a Datalog program-analysis engine

A security researcher has published a first-hand account of turning LLM agent memory into a miniature program-analysis engine, after growing frustrated with agents that lose track of findings during long vulnerability research sessions. According to the post on pwning.systems, which surfaced on the Hacker News front page, the experiment produced Lemmalog, a Datalog engine that lets an LLM agent maintain a consistent body of knowledge which updates itself when earlier observations turn out to be wrong.

The problem with transcript memory

The author reports spending several months using LLM agents for vulnerability research, where they proved effective at navigating large codebases, explaining unfamiliar subsystems and exploring potential attack surfaces. But once an investigation stretched over a few hours, the model drifted: it re-proposed approaches that had already been ruled out, forgot that an assumption had been disproven, and kept reasoning from observations that no longer held.

Standard memory systems did not fix this, the author explains, because they store what was said. Embedded conversations and retrieved observations amount to a transcript, and the model has to re-derive the current state of knowledge from it each time. The post gives a concrete example: if an attacker controls object_a, object_a points to object_b, and object_b is a kernel object, then the attacker can control a kernel object. Hours later, a debugger session may reveal that the pointer relationship rested on a mistaken assumption. A retrieval-based memory now holds both the original claim and its retraction, and the LLM has to sort out which conclusions still stand.

Recognizing program analysis

That structure looked familiar. The author's regular work involves program analysis, where a set of facts plus inference rules is iterated to a fixed point, and incremental techniques recompute only the affected results when an input fact changes. That is exactly the behaviour wanted from the agent: when an observation is retracted, dependent conclusions should become invalid automatically, rather than waiting for the model to notice the consequences.

The chosen formalism is Datalog, a declarative logic programming language in which you state facts such as controls(attacker, object_a) and rules that derive further facts, and the engine computes everything that follows. Because derivations are explicit, changing one input tells the system precisely which downstream conclusions to drop.

How the work is split

Lemmalog divides the investigation into a fuzzy half and a deterministic half. The LLM remains responsible for the messy part: reading natural language, source code and debugger output, and converting it into structured facts. Once the information is in fact form, the Datalog database, not the model, derives consequences and keeps them consistent.

Retracting facts turned out to be the tricky part, since a conclusion can have multiple independent supports. If a claim is derivable from two separate observations, removing one leaves it intact, while removing both makes it disappear. The post notes this matters for security work: a finding that a bug candidate is exploitable may survive the failure of one exploit primitive if another path still supports it. Lemmalog therefore tracks how each fact was derived and updates its support when inputs change.

Asking the agent why

That derivation bookkeeping yields a second benefit: provenance. When an agent concludes, after hours of work, that a candidate is exploitable, Lemmalog can answer why by returning the chain of observations and rules behind the conclusion. If an observation in that chain is later shown to be wrong, everything depending on it is invalidated automatically.

It also counters a common failure mode the author describes: models confidently asserting that something was already established when it was not. In Lemmalog, a conclusion without provenance simply is not part of the maintained state. The post acknowledges that this does not stop the model from hallucinating while extracting facts in the first place, but it confines the damage to the moment of extraction rather than letting it compound across a long session.

Why it matters

This is a practitioner's answer to a widely recognised weakness of LLM agents on long tasks: state drift. Rather than scaling context windows or tuning retrieval, it delegates consistency to a well-understood formalism from program analysis, letting the model do what it is good at — interpreting messy inputs — while a database handles consequence and retraction. For security work specifically, provenance turns agent conclusions from assertions into auditable claims with a checkable chain of evidence. The pattern also looks generalisable beyond vulnerability research to any multi-hour agentic workflow where facts get revised, such as debugging, incident response or large codebase migrations. And the post is a useful signal of where AI-assisted security is heading: not fully autonomous bug hunting, but hybrid systems in which an LLM's reasoning is grounded in an external, mutable knowledge base.