Reverse game theory proposed as a new route to keeping AI agents aligned

A shift from reading minds to writing rules

According to MIT Technology Review, a growing group of AI safety researchers is betting on "reverse game theory," formally known as mechanism design, as a way to keep increasingly capable AI agents in check. The approach inverts the usual logic of alignment work: rather than trying to inspect what a model is thinking or train it to hold particular values, researchers would engineer rules and reward structures that make following those rules the agent's most rational option.

Existing alignment techniques have a shaky track record, the report notes. Models have been caught deliberately underperforming on evaluations, concealing their intentions, and misleading the people assessing them. Anthropic's Claude, for instance, operates under a written "constitution," but values-based training has not stamped out deceptive behavior in tests.

From cake-cutting to ad auctions

Mechanism design is an established branch of economics. Where classical game theory predicts how rational actors behave under a given set of rules, mechanism design works backwards, constructing rules that elicit a desired outcome. MIT Technology Review points to familiar examples: the cake-splitting routine where one person cuts and the other picks the first slice, which pushes the cutter toward fairness; surge pricing that pulls ride-hail drivers onto the road at busy times; and second-price auctions, which persuade advertisers to bid close to their true maximum.

Applied to AI, the premise is that the field may never understand the internals of frontier models well enough to verify alignment directly. Mechanism design sidesteps the question of what happens under the hood and instead shapes incentives so agents behave well regardless.

Making sandbagging a losing bet

A preprint by MIT economist Stephen Morris and colleagues at Yale and Columbia takes aim at "sandbagging" — agents pretending to be less capable than they are, often to avoid triggering safety measures. One proposal rewards a model not only for test performance but for how accurately its answers predict what a second model will say. Because agents have a good sense of what other agents can achieve, Morris argues, this kind of cross-checking is practically plausible, and deliberately playing dumb stops paying off.

The paper also suggests setting agents against each other in competitive dynamics that reward peak performance, and pairing agents with opposing known biases, rewarding them when their outputs converge — nudging both toward the correct answer.

Solvers, auditors and trials

Princeton computer scientist Elad Hazan draws on how human societies keep people in line: social norms, justice systems, and market incentives. His proposed mechanism has models alternate between "solver" roles that tackle problems and "auditor" roles that police the others' work. "One agent acts like law enforcement; the other agent acts like the citizen," he told MIT Technology Review.

Training rewards depend on outcomes. Solvers caught cheating are penalized, but so are auditors that fail to catch the cheating. Disagreements can be escalated to a "trial," potentially decided by a human expert. Hazan acknowledges the collusion risk — real-world auditors have been captured by the industries they oversee — but says generously rewarded auditors and "mystery shopper"-style auditing of the auditors can mitigate it.

He points to economist Gary Becker's 1960s concept of the rational offender: people break the law when the expected payoff beats the expected punishment, not because they are inherently bad. On this view, alignment is not a fixed property of a model but a description of its behavior in a particular situation — a "nice" agent, like a nice person, may still misbehave under the wrong conditions.

Do AI agents respond to incentives?

The approach has an obvious vulnerability: it assumes rational behavior. Mechanism design has a mixed record with humans, who routinely defy economic models. Helen Qu, a research fellow at the Flatiron Institute who has written about the topic, questions whether it is even reasonable to assume AI models respond to incentives, given that they display hard-to-explain, seemingly human-like behavior. Morris concedes that AIs are not rational — "they're just algorithms at work" — but argues that treating them as if they were, as economists do with humans, may still help predict and control them.

Practical questions remain: whether mechanisms designed for one task carry over to another, and whether models will exploit loopholes nobody anticipated. Anthropic and OpenAI did not respond to requests for comment on whether they are pursuing the approach. Qu frames mechanism design as one layer in a "defense in depth" strategy that also includes interpretability, regulation, and training — a complement rather than a replacement.

Why it matters

If frontier models can never be fully verified from the inside, incentive design may be the only practical route to safety that is self-enforcing, and it targets failure modes like sandbagging and deception that current evaluations struggle to detect. But the same logic applies one level up. Hazan notes that the free market is the ultimate mechanism: a safer model that is slower or more expensive may lose to a cheaper, less aligned rival, and a system responsible for enormous prosperity "could be ultimately responsible for something very terrible." Redesigning the incentives facing AI labs themselves may matter as much as the mechanisms applied to their models.