How Claude's AI text watermark works: tournament sampling, detection and limits

The mark lives in the token choices

A watermarked language model leaves its signature not in hidden metadata but in the words it picks. According to a dev.to explainer, the resulting text reads normally, and the mark survives being copied into a plain-text editor because it is carried by the token sequence itself. A detector holding the matching secret key can test for the pattern later.

The occasion for the walkthrough is Anthropic's August 2026 post "How Claude's text watermark works". Per the dev.to piece, Claude uses a version of SynthID-Text, the scheme Google DeepMind published in Nature in 2024, and Anthropic says it is introducing watermarking to meet the EU AI Act's transparency requirements.

One word at a time

A language model writes one token at a time — a word, part of a word or punctuation — and assigns probabilities to every possible next token; GPT-2's tokenizer, for instance, contains roughly 50,000 of them. Unwatermarked sampling follows those probabilities directly. Watermarking leaves the model's weights unchanged and only influences which candidate is selected.

The author follows one decision: after "The cat", suppose only "sits" (50%), "sat" (30%) and "lies" (20%) have non-zero probability — invented numbers for illustration. Unwatermarked, the model might write "The cat sits on the mat"; watermarked, it might produce "The cat sat on the rug".

Tournament sampling

SynthID-Text picks the next token through tournament sampling. Several candidates are first drawn according to the model's probabilities — the example draws four and gets "sits, sat, sits, lies". An algorithm seeded with the secret key and the preceding tokens then gives each candidate a score of 0 or 1. Candidates meet in bracket rounds; the higher score wins each match, ties are broken randomly, and copies of the same token share a score within a round even though a token's score can differ between rounds.

In the worked example, "sits" loses both of its matches despite having the highest model probability, and "sat" wins the tournament. The crucial property is repeatability: the same key, settings and token context always produce the same scores, which is what makes detection possible later. Any single choice proves little, since an unwatermarked model could pick the same word, but across many tokens watermarked text tends to carry higher scores than chance alone would explain.

The quality question

Drawing candidates from the model's probabilities keeps likely continuations well represented, but that alone does not guarantee good writing. SynthID-Text ships a quality-preserving configuration and another that yields a stronger watermark at a cost in quality. According to the dev.to article, Google tested the quality-preserving configuration on nearly 20 million Gemini responses, and the paper reports no statistically significant difference in thumbs-up or thumbs-down ratings versus unwatermarked output. Detection is also hardest where the model has little real choice about the next token, because there is less room to leave a mark — a practical limit in passages where accuracy forces specific wording.

Detecting without the prompt

The detector needs the text, the secret key, the same watermark settings used at generation time and the matching tokenizer. Because the preceding tokens are already in the passage, it can recompute each token's watermark scores from context alone; with a four-token context it simply skips the first four tokens. A basic detector averages the recovered scores and skips repeated contexts so a repeated phrase is not counted as fresh evidence.

Unwatermarked text should average around 0.5, since the scoring functions assign roughly equal numbers of zeros and ones, while watermarked text trends higher. That average is not proof by itself: short passages can score high by chance, so thresholds are calibrated against unwatermarked text and must account for length. Editing erodes the signal — with a five-token window, replacing one token changes the score inputs at that position and the next four, and replacing a whole word can affect more tokens depending on tokenization. Small edits leave much of the mark intact; extensive rewriting can make it undetectable.

Avoiding a billion-way bracket

A tournament 30 rounds deep would require over a billion candidates per token, so the small implementation the author tested with GPT-2 in Google Colab computes the equivalent winning probabilities directly. Each round adjusts the probability distribution to favour tokens that scored 1, weighted by how much probability they already hold. In the cat example, "sat" climbs from 30% to 45% after round one and 69.75% after round two — a figure the source says reflects the chance that at least one finalist is that token.

Why it matters

Anthropic tying watermarking to the EU AI Act suggests provenance signals could ship by default in widely used assistants, giving platforms and regulators a technical hook for transparency obligations. But the mechanics set realistic expectations. Detection is key-gated, so it is not a public forensic tool anyone can run; verdicts are statistical and weaken on short, heavily edited or low-choice text; and the quality-preserving configuration Google tested at scale is the one likely to be deployed. Anyone relying on these detectors — publishers, educators, moderation teams — needs to understand both the false-positive calibration and how easily rewriting erases the evidence.