Fault-injection scan of 31 MCP servers finds only 3% with enforceable output schemas

Only 3% of tested tools have a meaningful output contract

A fault-injection audit of 31 popular MCP servers found that just 3% of the 265 tools examined declare an output schema strict enough to reject a corrupted response. The study was self-published on dev.to by Timur Rakhmatullin, who also built the testing harness, mcp-drill, used to run it.

The scan covered local reference servers as well as remote endpoints from well-known providers, including Microsoft Learn, Hugging Face, Cloudflare and DeepWiki. Of the tools measured, 56% declared no outputSchema at all, leaving clients nothing to validate against. Another 42% declared a schema the study classifies as effectively useless: when every leaf value in a response was swapped for corrupt or out-of-range data, the schema still validated. Only 7 tools out of 265, or 3%, rejected the tampered payload.

How the test works

MCP servers can publish an outputSchema describing what a tool's result should look like, giving clients an automatic way to check responses. The harness corrupts a response while keeping its structure and types intact — strings remain strings, objects remain objects — and then checks whether the declared schema catches the damage. Schemas that admit corrupted data are labeled vacuous; those that reject it count as enforceable.

The scan is deterministic and involves no language model, so according to the author each score reflects the server itself rather than whichever agent calls it. A second set of probes tests error handling by sending unknown methods, unknown tool names and calls with missing required arguments. On that front the ecosystem performed well: 30 of 31 servers returned a proper JSON-RPC or tool error. The weakness is confined to the success path.

Standout scores

Per-server results varied widely. git-mcp-server scored best at 18% enforceable. Hugging Face's remote server reached 12%, making it the only brand-name remote endpoint with any enforceable schemas. The filesystem reference server managed 7%. At the other end, the everything reference server, Microsoft Learn, DeepWiki, Playwright and desktop-commander all scored 0%, the last two because they declare no schemas at all. The author reports the pass rate held between 2% and 3% as the sample grew from 18 to 31 servers, suggesting the result is not a small-sample artifact.

One structural cause stands out in the post: the default behavior of FastMCP, described as the dominant Python SDK for MCP, automatically wraps a tool's return value as a generic object containing a single string. Such a schema validates nearly any output, and every server that keeps the default inherits that weakness.

Why it matters

Agent workflows routinely feed one tool's output straight into the next tool without a human reviewing the intermediate result. The only automatic checkpoint between "the transport succeeded" and "the payload is actually right" is schema validation. When schemas are vacuous, a truncated or silently wrong response — half a JSON document, or a success marker on a failed lookup — flows into the next step unchallenged.

The author is careful to separate this from security auditing tools such as mcp-scan, which ask whether a server can be abused. This study asks the inverse question: whether a server's results can be trusted when it operates normally. There are caveats worth noting — it is a single self-published study, and its author maintains the open-source tool built around the methodology. But the failure mode it documents is mechanically checkable: mcp-drill is Apache-2.0 licensed, installable via pip, and can scan any local or remote server, including as a CI gate through a GitHub Action.

What the study recommends

Server authors are advised to constrain values, not just shapes: enums, regex patterns, formats and numeric bounds are what make a schema reject bad data. Setting additionalProperties to false helps, but on its own is not enough, since a corrupted string still passes as a string. Tool consumers should not treat the mere presence of an outputSchema as a safety property and should validate results semantically downstream. And anyone reviewing MCP integrations should ask about enforceability specifically, because auto-generated schemas will push coverage numbers toward 100% while doing little to catch wrong answers.