Project Timestamper anchors Common Crawl's 339 billion web snapshots to Bitcoin

Project Timestamper has cryptographically timestamped the entire Common Crawl archive: 339 billion web page snapshots collected over 18 years of crawling and totalling more than 10 petabytes of data. According to the announcement on the project's blog, every capture has been committed through a chain of hashes to the Bitcoin blockchain, producing verifiable proof that each snapshot existed byte for byte in the collection as of 26 September 2026.

Common Crawl, which the post describes as perhaps the world's second-largest crawl after the continuous archiving performed by the Internet Archive's Wayback Machine, preserves pages exactly as they appeared at capture time. Project Timestamper frames the collection as a recording of a significant part of humanity's collective output over the past two decades, and argues that its integrity matters as AI grows more capable of generating plausible data and tampering with databases.

How the timestamping was done

Hashing and timestamping hundreds of billions of items individually would have been prohibitively slow. Project Timestamper instead built on work Common Crawl had already done: the foundation had hashed every web page snapshot and grouped those hashes into blocks of up to 3,000 index entries. That reduced the job to hashing roughly 134 million blocks, which the project completed in a few weeks before timestamping the resulting hashes in a single batch through the OpenTimestamps service, anchoring the commitments to the Bitcoin blockchain.

Because the on-chain record fixes the state of the collection at 26 September 2026, the announcement says, any alteration made after that date will be detectable.

Verifying a single snapshot

The hashes and timestamp attestations were arranged so that verifying any one of the 339 billion snapshots takes a few seconds and a few hundred kilobytes of downloaded data. The project provides a verification tool that checks a capture with a single command, run via npx tsx verify.ts against the Common Crawl block collection.

A sample verification shown in the blog post confirms a capture of the English Wikipedia main page, pinpointing the crawl segment it belongs to (CC-MAIN-2026-39), the attesting Bitcoin block number 968666, an attestation time of 2026-09-26T09:35:20Z, and the SHA-1 hash of the archived WARC payload.

Why it matters

Web archives increasingly serve as reference points for what the web said and when. As generative systems make imitation and confabulated content cheaper to produce, a proof of existence tied to a public ledger gives researchers a way to distinguish an untouched record from one modified after the fact. The timestamp does not certify that a page was captured accurately in the first place, only that the stored bytes existed in Common Crawl's collection at a known date, but that is precisely the guarantee an archive needs.

Common Crawl's open corpus is heavily used, including as training data for machine learning, so an integrity anchor on top of an already open dataset is meaningful for downstream users. Project Timestamper credits the Common Crawl Foundation for maintaining the collection with open APIs and open data, and says it hopes to keep timestamping new material as the crawl continues, which would extend the same guarantee to future captures.

The exercise also demonstrates that blockchain timestamping can scale when it builds on existing infrastructure: Common Crawl's pre-computed hash blocks turned a problem of 339 billion items into one of about 134 million, completed in a matter of weeks.