shutdown.fyi probes Russia's internet from both sides to separate censorship from outages

The team behind shutdown.fyi, an open monitoring project, has published a methodology for telling Russian internet censorship apart from ordinary service failures by probing the same hostnames from inside Russia and from abroad, then comparing where each connection attempt stops. As a dev.to post by the project's maintainer explains, complaint-based outage trackers cannot separate a service that is down for everyone from one that is up everywhere except Russia, because both produce an identical spike of user reports.

Probing from both sides of the border

The tracker covers 38 services, 8 carriers and 85 regions. Every five minutes it probes up to two real website hostnames per service, deliberately avoiding CDN image hosts, since a successful handshake there proves nothing about the site itself. Each probe walks four steps: DNS resolution, a TCP connection to port 443, a TLS handshake carrying the correct server name indication, and finally an HTTP response.

The same sequence runs from a server in Russia and from one in Helsinki, and the verdict comes from the difference between them. If both vantage points pass every step, the service is open. If the Russian probe stops at DNS, TCP or TLS while the foreign probe passes, the service is restricted, and the failing step points at the blocking technique. If both sides fail, the service itself is broken. And if the service answers Russian requests with HTTP 403 or 451, the refusal comes from the service, not from the state.

What the difference reveals

The post includes a live result for youtube.com captured on the day of writing: from Russia, TCP connects in about 20 milliseconds but the TLS handshake times out, while from Helsinki the full sequence completes with an HTTP 200. That shape — transport fine, TLS hanging, and only from the Russian vantage — is what filtering on the server name inside the TLS ClientHello looks like, according to the author. Telegram, WhatsApp and Instagram fail one step earlier: their IP addresses are unreachable at the TCP stage, a separate blocking method the project labels distinctly.

Two axes instead of one status light

Conventional status pages reduce health to a single coloured dot, and the post argues this misleads in both directions for long-running restrictions: an anomaly detector treats a service blocked for months as unremarkable, while a plain reachability check reports it as broken forever. shutdown.fyi therefore splits status into two independent axes. Access comes only from the active probes and exists only for services, since a carrier or region has no reachability of its own to measure. Incident, meanwhile, answers whether things are worse than usual for this hour, drawing on complaints, OONI measurements, IODA traffic data and BGP visibility.

The hourly framing matters because complaint volume at 3 a.m. and 8 p.m. differs by an order of magnitude. The baseline is the median of the same hour over previous days, with a floor at half the weekly median so that three complaints on a quiet night do not register as a tenfold spike.

Bugs that shaped the tool

Four engineering mistakes documented in the post shaped the current behaviour:

• Treating levels as counts. While aggregating eight public outage detectors, the project initially wrote a source's rolling 24-hour complaint level into every hourly slot, turning a reported 18 messages per day into roughly 450 on its chart. The fix converts a 24-hour level to one twenty-fourth per hour, and combines totals with chart shapes from sources that publish both.
• A bank that was never blocked. A major Russian bank was flagged as restricted because its anti-bot protection drops connections from hosting IP ranges and the probe runs in a datacenter. Domestic services on a government whitelist now receive a no-conclusion verdict when only the Russian vantage fails.
• Flapping incidents. Closing an incident on the first healthy snapshot filled the feed with six-minute outages. Incidents now close only after 15 minutes of normal readings, and shorter blips are hidden.
• Cyrillic word boundaries. A news matcher for the Russian telecom regulator matched inside a word meaning "emphasized", pushing a mayor's speech onto the status page, because the \b regex construct does not handle Cyrillic without explicit lookarounds.

Open data and acknowledged limits

All data is exposed as JSON without an API key under a CC BY 4.0 licence, including per-service probe verdicts, alongside an embeddable widget. Documentation, examples and the full methodology live in a public GitHub repository, Chumbayoumba/shutdown-fyi-open. The maintainer is explicit about the weakest link: a single Russian vantage point sitting in a datacenter. Residential probes inside Russia would make the verdicts considerably stronger, and the author invites feedback from others doing network measurement.

Why it matters

Censorship measurement tends to be either deep but opaque, as with private datasets gathered inside restricted networks, or broad but shallow, as with complaint spikes that conflate outages with blocking. The cross-border, step-by-step comparison at the heart of shutdown.fyi produces an attributable verdict: not merely unavailable, but unavailable at the DNS, TCP or TLS layer, only from inside Russia, which points directly at the filtering technique in use. Publishing that pipeline openly, with keyed JSON access and an honest accounting of false positives and unit bugs, lowers the barrier for researchers, journalists and engineers verifying claims about Russian internet restrictions, and it offers a workable template for measurement projects anywhere ground truth is hard to obtain.