Mano-CUA open-source browser agent beats Gemini 2.5 Pro and Claude 4.5 on WebRetriever benchmark

The benchmark numbers

A dev.to post from Mininglamp-AI reports that the group's open-source browser agent, Mano-CUA 1.1, scored 41.7 on WebRetriever Protocol I, a benchmark built around navigation, retrieval and form completion on real websites. On the same evaluation, Gemini 2.5 Pro Computer Use scored 40.9 and Claude 4.5 Computer Use 31.3, according to the post. The author describes the roughly ten-point gap to Claude as consistent rather than noise, recurring across multi-step forms, dropdown selection and paginated results.

Two caveats are worth noting. These are the project's own numbers, published in what is effectively its announcement post, not results from an independent lab. And the framing that GPT and Claude "failed" the task is looser than the post's own data: Claude's score is given, but no GPT figure appears anywhere in the text — GPT-4o is discussed only in general terms as weak at browser work.

Screenshots instead of DOM parsing

The architectural bet behind Mano-P is that browsers should be operated the way people operate them: by looking. The model reads screenshots, locates clickable elements from pixels, and decides the next action from visual context alone. There is no DOM inspection, no accessibility-tree dependency and no site-specific hooks. The argument is resilience: when a site's markup changes but the visual layout does not, a DOM-based agent breaks while a pixel-based one keeps working.

General-purpose models, by contrast, typically parse the DOM or lean on accessibility APIs, an approach the post says holds up on simple pages and collapses on JavaScript-heavy apps with dynamic rendering and custom components.

Form filling is the stress case. According to the author, general models lose their place after four or five interactions, target the wrong field, miss required inputs or repeat a failed action in a loop. Mano-CUA is built to check the outcome of every action before proceeding — if a dropdown fails to open it clicks again, and if a field rejects its input it reads the on-screen error and corrects course.

Local execution and open weights

Unlike the computer-use offerings from the major labs, the model runs entirely on-device. The post cites roughly 80 tokens per second on Apple M5 Pro hardware with the 4B quantized model, and recommends a Mac with Apple silicon and 32GB of RAM. With the local flag, screenshots never leave the machine; without it, inference routes through Mininglamp's cloud endpoint, which is faster but sends screenshots to a server. The code, CLI tool and model weights are published under Apache 2.0 on GitHub as Mininglamp-AI/Mano-P.

Training and wider benchmark context

The model was trained in three stages: supervised fine-tuning on GUI interaction data, offline reinforcement learning over logged trajectories, and online reinforcement learning against live interfaces. The author considers that final stage decisive, because the model experiences what happens when a click is wrong and learns to recover — something static datasets cannot teach.

On the broader OSWorld desktop benchmark, a 72B evaluation version of the model reportedly reached 58.2% task success, 13.2 points ahead of the next specialized model. The post is careful to note this is not the model users run locally: the 4B variant scored 56% on a 100-task macOS GUI test, against 39% for Qwen3-VL-Plus on the same set.

Why it matters

For teams building agent products on top of browser control, this is a concrete data point in the specialization debate. Frontier general models are convenient, but the reported numbers suggest they still trail purpose-built, GUI-trained agents on precisely the vertical — multi-step web interaction — where reliability matters most. The margin over Gemini is thin at 0.8 points, so the honest reading is that specialized agents are at least competitive and sometimes far ahead, not that they have run away with the category.

Two practical consequences stand out. Open weights plus a local runtime mean screenshots of internal dashboards, customer records or admin panels can stay on-premise, removing a major privacy objection to agent-driven automation. And because the stack is Apache 2.0, teams can evaluate it against their own workflows rather than taking a vendor's word. The caveat remains: this is one self-reported benchmark from the project itself, so reproduce it on your own tasks before replatforming.