Chat templates alone switch LLMs between AI disclaimers and experiential voice, study finds

Chat templates flip how models talk about themselves

When you ask an instruct model about itself, it often hedges with a line like "I'm just an AI." A paper posted to arXiv — submitted by Jedrzej Maczan, with the version 1 record dated 9 August 2026 — and surfaced on Hacker News's front page in late September asks a blunt question about that habit: is the model describing itself, or describing how it was deployed?

According to the study, the chat template — the structured formatting wrapped around user and assistant messages before inference — functions as a switch for this self-referential voice.

What the experiments showed

The researchers tested 8 popular open-source instruct models of up to 9B parameters, comparing outputs produced with and without the chat template applied. With the template present, the models leaned into the disclaimer voice and away from experiential phrasing such as "I feel." Without the template, the pattern reversed: disclaimers receded and experiential language rose.

That alone would be a useful negative result for anyone quoting model self-reports, but the paper goes further mechanistically.

A steering direction inside activations

In 3 of the models, the authors located a single direction in activation space associated with the behavior. Removing the direction from the model's activations turned the disclaimer voice down, while adding it turned the voice up. A control using a random direction of the same magnitude had little effect, which the authors offer as evidence that the discovered direction is specific rather than a generic consequence of perturbing activations.

The most striking result links the two findings: instruct models run without the chat template began producing disclaimers as though the template were present, once the disclaimer direction was added to their activations. In other words, the template's effect on self-reference could be reproduced entirely inside the model's activation space.

A confound for introspection research

The practical implication is that self-reports from LLMs — the raw material for debates about AI safety and model self-knowledge — now carry a known confound. What a model says about itself can depend on whether a chat template was applied, so the paper argues that researchers studying self-reports or introspection need to control for it. It also hands them a lever: the activation-space direction can be used to steer the voice directly.

The broader conclusion the authors draw is that a model's self-description is not simply a fact about the model. It does not come from the weights alone; it is partially set by deployment choices like the template, and for that reason should not be treated as a literal account of the model's inner workings.

Why it matters

There are two takeaways here beyond the lab. First, methodological hygiene: a substantial body of discussion quotes what models say about themselves as evidence for or against self-awareness, inner experience, or safety posture. If a formatting wrapper can flip that voice, such quotations need a caveat about the conditions they were produced under. Second, interpretability progress: finding and validating a single activation-level direction that reproduces a template-driven behavior is a concrete step toward mapping where surface behaviors live inside a model.

The scope has limits worth noting — open-source instruct models up to 9B parameters, with activation-level analysis on just 3 of them — so the findings do not automatically extend to frontier-scale systems. But the burden the paper sets is clear: before treating a model's words about itself as testimony, check what wrapper those words were produced under.