deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Bulk test of 37,000 MCP servers: 89.6% respond, a third require keys

A company running an MCP directory connected to every remote server in the official registry the way a real MCP client does. Of 37,383 tested, 89.6% answered, but a third demanded keys first and 12.6% of the registry looks like spam.

Bulk test of 37,000 MCP servers: 89.6% respond, a third require keys

The official Model Context Protocol registry has grown past 62,000 server entries, yet most published advice about it draws on hand-picked lists of a few dozen. NestDaddy, a company that runs its own MCP server directory, took a different approach: it connects to every remote server in the registry the way an MCP client does, every day, and records what happens. Its first full run, published on dev.to, probed 37,383 servers with remote endpoints and offers one of the first broad empirical pictures of the agent tooling ecosystem.

How the test works

According to the report, each check mirrors what Claude, Cursor or VS Code do when a user adds a server: it sends initialize over Streamable HTTP (or opens the stream for the older SSE transport), sends notifications/initialized, then requests tools/list. No credentials are ever sent, so a server replying with HTTP 401, 403 or 429 is counted as live but gated behind a key. Every request has a 10-second timeout, and all checks run from a single machine in Europe, meaning response times include the trip across the internet.

Nine in ten servers answer

Of the 37,383 remote servers tested, 33,496 — 89.6% — responded, a result the report calls better than expected for a registry anyone can publish to. The 3,555 failures split into 1,156 connection problems (DNS, refused connections, TLS errors or timeouts), 997 HTTP 404s, 230 responses with 503, 173 with 400, 170 with 405, 113 with 502, 105 replies that were not valid MCP initialize results, and 104 servers demanding payment before doing anything. A 404 or 405 usually means the registry entry points at the wrong path, such as the site root rather than the actual MCP endpoint, which NestDaddy flags as something maintainers should verify.

Keys, latency and tool bloat

Two findings cut against usability. First, 12,447 servers — about one in three — answered with 401 or 403, refusing to say anything before authentication. The report argues this carries a real cost: directories cannot list the server's tools and an agent cannot judge whether it is useful. NestDaddy recommends the friendlier pattern of answering initialize and tools/list openly and requiring the key only on tools/call, so users can see what they get before signing up.

Second, the ecosystem leans context-heavy. Across the 20,241 servers that exposed their tools, the probe counted 292,447 tools in total, with a median of seven per server; 815 servers offer exactly one tool and 4,325 offer twenty or more. Every tool name and description is sent to the model in every conversation whether or not the tool is used, so five servers with 30 tools each preload 150 tool descriptions into context — a token cost the report says also makes models worse at picking the right tool.

Latency, by contrast, looks healthy: a quarter of servers answered initialize in under 320 ms, the median was around 500 ms, 75% came in under 780 ms and 90% under 1.2 seconds, with only 842 servers slower than two seconds. Measured from Europe, the numbers flatter nearby servers, but NestDaddy reads them as evidence that the protocol itself is not the bottleneck — a server that takes seconds to say hello is usually cold-starting or doing work it should defer.

One in eight registry entries is spam

Before testing anything, NestDaddy hid 7,839 entries — 12.6% of the registry — as clearly mass-published. One GitHub repository sat behind 2,356 published servers, two more repositories accounted for a further 2,200, and some descriptions appeared word for word across more than 150 entries. Bulk publishing is not always spam, though: one publisher runs 1,710 live servers from a single gateway, one per public API, each sharing the same 12 base tools on top of its own. The registry is also growing fast, with 17,515 entries published in October and 16,819 in the first 13 days of November alone, and the report expects the spam share to grow unless registries add stronger checks.

Why it matters

This is one of the first large-scale mechanical measurements of an ecosystem that agent builders now depend on, and its practical takeaways are concrete: prefer servers that show their tools without a key, keep tool lists short and clearly described, and confirm that a published URL points at the real MCP endpoint rather than a homepage. The caveats are real — the data comes from a single company that sells its own MCP directory, and everything was measured from one European vantage point — but because the directory re-tests every server daily, the interesting signal going forward will be the trend line: whether the registry gets cleaner or noisier as it keeps growing.

  • #mcp
  • #model-context-protocol
  • #ai-agents
  • #developer-tools
  • #testing

Related posts