Crawler claiming to be ClaudeBot probed a site's .env file 20 times in one week

A familiar name asking for an unusual file

A small Japanese site received twenty requests for its /.env file in a single week, all from a user agent identifying itself as ClaudeBot, Anthropic's web crawler. The operator, writing on dev.to, notes that the file — which typically holds database passwords, API keys and cloud credentials — correctly returned a 404, but the request pattern stood out against otherwise ordinary crawler traffic.

The site's dashboard for the seven days ending September 18 shows what normal looks like: 27 front-page requests from Bytespider, tag pages fetched by meta-externalagent, a pocket wifi article read by ChatGPT-User. Two entries did not fit. The path /.env drew 20 requests attributed to ClaudeBot, and a nonexistent /read-document path drew 6 from PerplexityBot. Neither path exists on the site.

The dashboard could not answer the obvious question

Whether those twenty requests actually came from Anthropic is the question that matters, and the author's own tooling cannot resolve it. The dashboard marked ClaudeBot as 68% "verified" for the week — meaning roughly a third of its requests could not be confirmed as Anthropic's — but verification is computed per crawler, not per path. There is no way to tell whether the /.env hits landed in the verified or the unverified share. The data comes from AILYS Lens, a WordPress plugin the author develops, and the author also discloses a related diagnostic service called AILYS Doctor.

The post is careful on attribution: nothing in the logs should be read as a claim about Anthropic, whose crawler name was used without the company's involvement.

GreyNoise documented a broader spoofing campaign

Context comes from the security firm GreyNoise, which on August 31 published observations covering July 28 to August 23: 824 IP addresses running scans that forged AI crawler names. Six crawler names belonging to four AI companies were impersonated, with two Amazon crawler names appearing in even greater volume, and the requested paths were /.env, /.env.production, /.env.bak and /.aws/credentials.

The giveaway, as the dev.to author highlights: zero robots.txt requests across the entire campaign. The genuine ClaudeBot fetches robots.txt constantly — the author puts it at roughly 12% of its traffic — while the impostors never fetched it once. A client that hunts for configuration files but never checks the rules file is behaving like a credential scanner, whatever its header says.

A user agent is a claim, not an identity

The choice of crawler names was deliberate, the author argues. Site owners have spent the past two years being told to make allow-or-block decisions based on names like GPTBot, ClaudeBot and PerplexityBot, so firewalls and plugins now treat those strings as decision inputs. A user agent header is written by the client about itself, and the author demonstrated the point on their own site two weeks earlier: a single curl request with a spoofed ClaudeBot header was logged as ClaudeBot in the dashboard. A rule keyed to a string anyone can type is a convention dressed up as a control.

What operators can do

The post's recommendations, in order of importance:

• Never serve secrets over HTTP. If .env is unreachable, the scan is a wasted 404 and nothing more.
• Check addresses, not names. OpenAI, Anthropic, Perplexity, Common Crawl and Google all publish IP ranges or reverse-DNS schemes; a request claiming to be one of them and failing to match is worthless as evidence.
• Watch for what is missing. A client that never fetches robots.txt but does request config paths has a cheap and obvious signature.
• Treat crawler traffic as ordinary traffic rather than a separate, presumed-benign category.

The author also flags the limits of the data: twenty requests on one site is a small sample, consistent with the GreyNoise campaign but not proof of it, and page caches that answer requests without hitting the application mean every count is a lower bound.

Why it matters

Two trends converge here. Operators increasingly manage AI crawler access by user-agent name, and scanners are now deliberately forging those names to probe for exposed credentials. The incident itself is small, but GreyNoise's 824-address campaign suggests the technique is in active use at scale. The practical takeaway is structural: user-agent allowlists are conventions, not controls, and the defense that actually counts — keeping secrets off the HTTP surface entirely — costs nothing to implement.