· via Hacker News – Front Page (native)
Dragos seeded two million AI prompts with canary tokens and found three breached providers
Dragos sent roughly two million prompts carrying trackable canary secrets to AI services and traced where they surfaced, uncovering three breached providers, resellers logging prompts, and privacy settings that went ignored.

How the canaries were planted
Security researchers at Dragos spent five months seeding unique, trackable secrets into roughly two million AI prompts, then watching where those secrets turned up. The findings, published by a Dragos research team led by Chief Research Officer HD Moore, document breaches at three AI hosting providers, prompt retention by marketplace resellers, and privacy settings ignored in practice.
Between July and November 2026, the team submitted about two million prompts containing unique URLs and canary token credentials — values that no legitimate system had any reason to touch — to widely used AI services. Because each token was one of a kind, any later attempt to use one showed the prompt had been read somewhere along the supply chain.
The goal was to answer a question customers rarely can: what happens to prompts after they leave your environment? Providers routinely keep prompts and responses for operational, training, safety, or business purposes, and a single request can pass through model developers, cloud operators, inference platforms, and resellers before it is answered. Dragos argues the prompts and replies themselves often carry more value than the compute that processes them, since they routinely hold source code, internal messages, strategic plans, customer records, and credentials.
Three breached providers
The most serious discovery was outright theft. According to Dragos, attackers were actively siphoning request data from three hosting providers, and none of the three realized they had been compromised until the researchers notified them. The earliest activity dated to mid-July.
In the largest case, a couple hundred seeded AWS keys — each sent in exactly one request to a single provider, clustered in short windows during late October — were all stolen. Some came from requests the provider rejected before any model ran, which places the attacker at the provider's entry point rather than deeper in the inference stack. In the other two breaches, some of the stolen requests carried a Zero Data Retention flag, the setting meant to keep prompts from being stored at all.
Once the keys were in hand, the attacker followed a familiar playbook: testing which keys were live, sometimes within hours; enumerating S3 buckets and searching Secrets Manager and Parameter Store; then enabling hosted models in AWS Bedrock, minting Bedrock API keys, and adding fresh IAM users to retain access. Across the study, the bait keys were exercised in tens of thousands of AWS API calls, and one key seeded on October 21 was still in use three weeks later.
Dragos ties one adversary to two of the providers simultaneously — secrets sent to both were used together, almost immediately, from a single rented server — and observed its tooling shift over three weeks from a script posing as Chrome on Windows, to Boto3 running inside OrbStack, to the AWS CLI on Kali Linux. On November 1 the same actor brought in ARTEX, an LLM-driven penetration testing framework, though it did little here beyond confirming that already-stolen keys still worked, under user agents including artex-agent.
Resellers can read the traffic
Apart from the breaches, multiple resellers on an AI inference marketplace could view and retain customer prompts using ordinary logging — no vulnerability needed, no control bypassed. Dragos explains that routers, which expose a single API endpoint backed by a rotating pool of models and hosting providers, leave every intermediary along the route positioned to read the complete request and response. Deeply discounted sellers can fund their prices through unmetered consumer subscriptions or startup credits, and shadier marketplaces include inference billed to stolen AWS credentials or consumer API keys. A seller who wants customer data can post popular models at the cheapest rate, cover the gap themselves, and harvest the data from platform logs.
Privacy settings that don't hold
Dragos also documented providers disregarding customer privacy settings or their own published policies. Prompts submitted to U.S.-based providers were transmitted to Chinese AI organizations; unique URLs were accessed months after a single inference request; and one seeded link eventually surfaced in an internal company chat in China. Zero Data Retention is supposed to be the strictest option, and routers are expected to forward such traffic only to providers offering matching guarantees — but the researchers found the settings are frequently ignored, whether deliberately or through processing bugs, and a privacy flag offers no protection once an attacker is already inside.
Why it matters
For defenders, the study works as both a warning and a playbook. Any prompt sent to an AI service should be treated as data crossing your perimeter, traveling through parties you cannot audit under retention rules you cannot verify; on this evidence, privacy flags and ZDR commitments are best-effort at best. The canary technique itself is a cheap countermeasure worth copying: seed unique URLs or credentials into AI workflows and alert on any hit, because legitimate systems should never touch them. Credentials bound for an AI service deserve the same scoping and rotation as any exposed secret. And the ARTEX sighting, however small its impact here, is an early signal that attackers are bolting LLM-driven tooling onto routine credential theft.
- #security
- #ai
- #canary-tokens
- #supply-chain
- #llm