· via Hacker News – Front Page (native)
Google Search hides destination URLs behind opaque google.com/goto redirects
Google Search is replacing direct destination URLs in its HTML with opaque google.com/goto redirects, forcing scrapers to resolve each result through an extra request to Google.
What changed
Google Search has stopped putting destination URLs directly in the HTML of its results pages for many users, according to a blog post from Autom, a company that sells search-scraping APIs. Organic results now point at google.com/goto?url=…, an internal redirect that forwards the click to the real page.
Autom says the behaviour became consistent across searches in late August 2026 for signed-out sessions and private browsing windows. The company stops short of calling it permanent — it may still be an experiment — but notes it has spread well beyond an initial sliver of results pages, and that in logged-out sessions essentially every result URL now takes this form. The post circulated on Hacker News's front page.
How goto differs from earlier redirects
Google has wrapped outbound links before. Its long-standing google.com/url?q= format still left the destination visible as a plain, URL-encoded string in the query parameter, so anyone parsing the HTML could read where a result led.
The new format closes that loophole. According to Autom, the href on each result is now /goto rather than the destination, and the value in the url= parameter is not a straightforward base64 encoding of the target address — it appears to act as an opaque handle into Google's own index record for that page. There is no way to decode it locally.
The only dependable route back to the real address is to request the /goto URL and inspect the Location response header without following the redirect. Autom also points out that Google still needs the destination to render the results page itself — the domain, favicon and site attribution are derived from it — so copies of the URL survive elsewhere in the markup. That is a weaker, separate signal from the Location header.
The motive: making harvesting expensive
Autom reads the change as one more move against automated collection of search results, especially AI crawlers and SEO tools that pull large volumes of result links to assemble rival indexes. When destinations sat in the HTML, a scraper could harvest thousands of URLs from a single page load and never contact Google again. Under the goto scheme, recovering each destination demands a fresh request back to Google.
That changes the economics. Every resolved link is slower and noisier for the scraper, and each request gives Google a behavioural signal — one client asking to resolve hundreds of links in sequence is easy to spot. Autom situates the shift inside a broader campaign: the removal of the &num=100 parameter that once returned a hundred results per page, tighter BotGuard and SearchGuard checks, and litigation against SerpAPI.
Scraping vendors are adapting
Autom first saw goto links on a small share of results pages, too rare at that stage to build a dependable fix around without risking breakage for other users. With the wider rollout, the company has rebuilt its Google Search pipeline to resolve goto links by reading the Location header without following the redirect, and to return the final destination in the same structured fields its API already exposes. Customers should see no change in their integrations, though Autom says it will keep adjusting if Google alters the format again.
Why it matters
The visible impact on ordinary searchers is small — clicks still land on the right page, and the site's name and favicon are still displayed. The structural impact is larger. In one of the web's main navigation surfaces, the underlying HTML no longer states where a result leads; finding out means asking Google, one link at a time, and each ask identifies the asker.
For anyone building on search data — AI search startups, SEO tooling, researchers, archivists — the cost of assembling an independent index from SERPs has risen sharply, which appears to be precisely the intent. It also concentrates more control over the link graph inside a single company: Google now sits as an intermediary even between its results page and the destinations it lists. Whether this stays a test or becomes the default, it signals where Google is heading as AI-era scraping pressure grows.
- #google-search
- #web-scraping
- #serp
- #redirects
- #search-engines