BetterWebSearch MCP ships keyless web search and slimmer agent contexts

A developer has published BetterWebSearch MCP on dev.to, an open-source Model Context Protocol server that gives AI agents web search, page extraction and research tooling with no API key required in its default setup. The server searches through DuckDuckGo out of the box, with Brave Search and Tavily available as optional providers, and the project is MIT-licensed.

Keyless by default, local by design

The server runs locally over MCP's stdio transport and, according to the post, puts no vendor-hosted proxy, telemetry or analytics layer in the path: requests go straight to the search provider and the target websites. Starting it takes one command, npx -y better-web-search-mcp, or a short entry in a client's MCP configuration, with no environment variables needed for the default mode. The developer says it works with any stdio-capable MCP client, naming Claude Desktop, Claude Code, Cursor, VS Code Copilot, Windsurf, Cline, Zed and OpenCode.

Extraction that escalates only when needed

A plain HTTP request often comes back nearly empty on modern sites because the content is rendered by JavaScript. BetterWebSearch answers this with a three-stage pipeline: a fast plain fetch first, then structured data embedded in the page, such as JSON-LD and framework hydration payloads like __NEXT_DATA__ and __NUXT__, and finally a full Playwright browser session with network interception. It moves to the next stage only when the previous one fails, so lightweight pages stay fast while JavaScript-heavy apps still render. The server also records which strategy succeeded for a given domain, letting later visits skip stages known not to work.

Research folded into one call

The exposed tools include web_search for multi-provider search with deduplication and reranking, web_extract for clean content from a single URL, web_find for searching within one site, web_news for recent news with diversity filtering, and web_research, aliased as deep_search. The flagship is web_research: instead of an agent chaining a search with five separate extraction calls, the server performs search, extraction and passage selection itself and returns a compact set of cited passages. Two supporting features shape those results. Query expansion rewrites one query into several related searches — the post's example turns a question about unlimited mobile data in Germany into German-language and alternative phrasings — and content clustering collapses near-identical articles so a republished wire story is not cited as five independent sources.

Self-reported benchmark numbers

Across 12 live-web research questions, the author compared a traditional workflow (search, then open the top five pages) against a single web_research call. The traditional path returned 820,229 characters, roughly 205,000 estimated tokens, over 137.3 seconds; the research call returned 110,973 characters, about 28,000 tokens, in 49.6 seconds. That is an 86.5 percent overall reduction in returned text, with a median per-question reduction of 83.8 percent. A second test checked whether answers survive the compression: on 34 questions with known factual markers covering numbers, HTTP status codes, acronym expansions and general facts, 33 of 34 markers were retained, a 97.1 percent rate. The benchmark code ships in the repository and can be run with npm run bench. These figures come from the tool's own author rather than an independent evaluation.

Guardrails for untrusted web content

Because agents consume fetched pages automatically, the server treats web content as untrusted input. URLs are validated against SSRF targets — localhost, private IP ranges, link-local and reserved addresses are rejected — and redirects are rechecked at every hop. On the prompt-injection side, web_extract can flag text that looks like an attempt to steer the agent with embedded instructions, attaching the warning to the result without rewriting the original content.

Why it matters

API keys and paid search plans are persistent friction in agent development, and a keyless DuckDuckGo default removes the setup step that most search MCP servers demand. The larger contribution may be context economics: if a five-page lookup really costs on the order of 200,000 tokens, naive retrieval is impractical for long agent sessions, and pushing passage selection into the server is a plausible fix — though the retention claims deserve outside verification. The security posture also stands out, since retrieval-augmented agents that blindly trust fetched pages are a growing attack surface. As a single data point from its own author, the results warrant skepticism, but the design choices — escalation-only extraction, citation clustering, injection flagging — target real, recognized problems in agent retrieval workflows.