canimcp audits MCP tool schemas for quirks that silently break Claude Code and Codex

A CI check for silent MCP client failures

A developer has released canimcp, a zero-dependency Node CLI that checks an MCP server's tool schemas against a corpus of documented client bugs and reports which client, at which version, breaks on which tool. Writing on dev.to, the author frames the problem with a real case: a tool discussed on r/mcp stayed invisible to Claude Code for 17 days because its input schema used a root-level oneOf. Nothing errored on the server, nothing errored on the client — the tool was simply absent from the client's list.

The quirks behind it

The corpus is seeded from a comparison published by SineFrame, which ran Claude Code and Codex against the same ten MCP servers at pinned versions. According to the dev.to write-up, five of the ten features tested behaved differently across the two clients, and no single feature tripped both. Among the specific incompatibilities:

• A tool argument named ids[] makes Claude Code drop the tool without any warning.
• An input schema of roughly 14 KB gets compacted by Codex, and required nested fields are lost in the process.
• A server exposing 2049 tools has its list truncated by Codex to the first 2048.
• A root-level oneOf or anyOf sitting next to properties in the input schema causes Codex, checked at 0.162.0, to lose the property names, so the call arrives at the server mangled. The recommended fix is to move the composition keyword inside each property definition.

How canimcp works

canimcp accepts either a saved tools/list response or a server startup command, walks the tools it finds, and prints per-tool findings graded as errors, warnings or advisories — a sample run against a ten-tool fixture produced eight findings. Every finding names the affected client and version, a suggested fix, a source reference, and an evidence grade: measured, probe, historical or field-report. The author says the grading exists because parts of the underlying matrix come from direct binary probes that have not yet been re-run under a fuller harness, so a probe result should be treated as a hint rather than a confirmed verdict. The tool runs via npx on Node 18 or newer and is published on GitHub as princejain756/canimcp.

A corpus, not a validator

The author is explicit that this is not a schema validator, and describes what changed their mind: in Codex's code mode, the model never sees the JSON Schema at all, only a compacted TypeScript declaration generated from it. So canimcp is really answering what each client actually sees, with the schema being merely the part that can be inspected statically.

It is also not a live test. It examines the schema, not runtime result shapes or whatever client build shipped today. For real client trials the author points to a separate harness called M3, positioning canimcp as a quick gate to run in CI beforehand. Coverage is deliberately conservative: Cursor, Gemini CLI and other clients are not seeded into the corpus yet, and pull requests adding entries must include a reproduction and the client versions involved.

Why it matters

MCP's promise is write-once, run-in-any-client interoperability for agent tooling, and silent breakage undermines it in the worst way. A dropped or mangled tool looks like a server bug, and neither side's logs say otherwise — the r/mcp case took 17 days to diagnose. Because the failures are client-specific, version-pinned and non-overlapping, per the SineFrame data, server authors cannot reason about them from first principles; they need exactly the kind of empirical, evidence-graded corpus that canimcp packages. Moving that check into CI turns an invisible class of failure into a build-time report, before users hit it in production.