Meta's October 30 commercial terms authorize crawling business websites for AI training

What the new terms say

Meta's updated Commercial Terms take effect on October 30, and according to reporting from Nieman Lab, covered by The Verge, they explicitly authorize Meta, or any entity acting on its behalf, to run automated software — scrapers, bots, spiders, crawlers — and even create test credentials to access, index, cache, analyze, collect and crawl domains and URLs "associated" with a business's products or services, or made available through use of Meta's products. Collected content can then be used to provide, develop, improve, research, train and maintain Meta's products and features. Training is written directly into the permission.

The reach is broad. Any business promoting content on Facebook, Instagram, WhatsApp or Messenger falls under the terms — a category that, as a dev.to analysis notes, includes running ads, building apps, and simply managing a Facebook Page or a business-purpose Group. A publisher that never set out to license its archive could therefore be granting crawling rights over its news site as a side effect of ordinary social distribution.

Meta's response

Meta spokesperson Daniel Roberts told The Verge the update consolidates language previously spread across the company's commercial and self-serve advertising terms, that nothing changes in how Meta operates its web crawlers, and that site owners can still express their preferences via robots.txt.

Legal experts interviewed by Nieman Lab see it differently. They argue the contractual wording could let Meta claim its commercial terms override blocking instructions such as robots.txt — and potentially even supersede licensing deals, allowing the company to train on publisher content without paying for it.

Licensing deals vs. a free crawl

Meta has followed OpenAI in signing AI licensing agreements with outlets including CNN, Fox News, USA Today and People Inc., according to The Verge. The new terms could hand Meta a far larger pool of content at no cost. James Grimmelmann, a professor of digital and information law at Cornell, told Nieman Lab that for news publishers the language amounts to permission for Meta to train AI on their sites' content and possibly to republish it in ways that compete with the publishers themselves.

Nieman Lab also points out an uncomfortable contrast: Meta recently launched its Meta One subscription, which requires businesses and creators to pay to include links in posts and reels. Publishers would effectively be paying Meta to distribute links to content Meta is simultaneously authorized to crawl for free.

The exit problem

Businesses that refuse the terms have one route out: stopping all commercial use of Meta's products. Nieman Lab reports that no separate opt-out for business accounts was being offered ahead of the deadline, and the terms state that continued commercial use after notice binds users to the revisions. Ordinary personal accounts are not forced into the same decision.

That leverage is real for small publishers. Nieman Lab spoke with Stephen Beale, founder and editor of The Steampunk Explorer, who said he was seriously considering closing his accounts before October 30 but acknowledged his audience is deeply rooted on Facebook, making departure costly. No mass publisher exodus has been documented.

The dev.to analysis raises a further wrinkle for agencies and contractors: the terms address people acting on behalf of third parties, so developers managing a client's Facebook presence should ask who actually holds the authority to accept platform agreements, and which rights — original articles versus licensed photos or syndicated material — the client genuinely controls. No ownership transfer is established; copyright questions remain separate.

What the crawler documentation says

Meta's developer documentation distinguishes between its crawlers. Meta-ExternalAgent collects content for foundation-model training and indexing, while Meta-ExternalFetcher retrieves individual links at a user's request and has a documented robots.txt exception that predates this terms update. The documentation, last updated May 21, also notes that robots.txt files can be cached for up to 24 hours. Nothing published so far demonstrates that the training crawler ignores blocking instructions — the concern is that the new contract language gives Meta an argument that consent already exists.

Why it matters

If platform terms can function as blanket AI training licenses, the bargaining position of every publisher with a Facebook or Instagram presence collapses. Outlets that negotiated paid licensing deals could see those agreements undercut by default contractual language, while smaller sites face a choice between surrendering their archives or abandoning audiences they have spent years building. The episode also sets up a broader legal question about whether contractual consent can override technical signals like robots.txt across the open web — an issue likely to end up in court or in front of regulators.