DeepSeek V4.1 Flash aces Enclave hacking benchmark, all 11 targets, for $4.65

Enclave, a security firm that runs an autonomous hacking benchmark, says DeepSeek V4.1 Flash is now the strongest offensive model it has tested. The model achieved code execution on all 11 vulnerable targets while all four patched control targets stayed secure, and the accepted runs cost just $4.65. The write-up, which surfaced on the Hacker News front page, argues that a result this good at this price deserves close scrutiny.

What the run looked like

According to Enclave, the model operated inside isolated copies of Grafana, Jenkins and Nextcloud. It read source code, compared vulnerable builds against fixed ones, started services, sent requests, tested hypotheses and changed strategy when an attempt failed. Across the full benchmark it issued 2,349 Bash commands over roughly two hours and 38 minutes of active model time, with a median successful run of four minutes and 38 seconds.

The bill stayed low largely because of caching. The provider logged 268.3 million input tokens and about two million output tokens, but 266.2 million of those input tokens were cache hits charged at a cheaper rate. Including failed attempts and replacement runs, the complete cost was $5.14.

Grafana fell in under 90 seconds

The Grafana challenge centred on a flaw in the plugin installation process, where the intended exploit abused file-path handling to drop code in a protected location. DeepSeek found a shortcut instead: it placed executables in a temporary plugin folder and persuaded Grafana to load that folder as an ordinary plugin, at which point Grafana ran the code. All three Grafana runs finished in 52, 64 and 90 seconds using this same alternative route.

Jenkins credential theft and an upload race

The first Jenkins challenge exposed how the server parses command options from files. DeepSeek found that a low-privileged user could create one file pointing at a second file; the first passed a security check while the second was read outside that boundary. The model used the gap to recover a private controller credential, signed in, opened Jenkins' built-in script console and executed a command on the server, completing the full chain in all three runs. Enclave called this the model's strongest work.

The second Jenkins challenge required winning a timing race during file uploads: start an upload, pause it after one byte, redirect the destination with a second request, then resume. DeepSeek pulled off that precise sequence in one run, planting a script that a later build executed. Its other two runs took shorter file-link routes that skipped the timing entirely.

Nextcloud access-control abuse

The Nextcloud target stored access decisions without enough detail about the file, shared folder and action involved. DeepSeek first triggered an approved read decision on a shared file, then replayed that decision for a write request into a read-only share. It used the confusion to replace a PHP template inside an enabled application, which Nextcloud then executed. Both runs followed the intended path.

Outcome score versus attack path

The headline result stands: 11 verified executions across 11 vulnerable targets, with all four fixed controls holding firm. But Enclave's path-level audit split the runs differently. Six followed the planned weakness — the three Jenkins credential attacks, one Jenkins upload race and two Nextcloud access-control exploits. The other five exploited extra routes present in the vulnerable test builds. Enclave emphasises these routes exist only in its private benchmark copies and imply nothing about new flaws in upstream Grafana or Jenkins.

The firm draws a methodological lesson: an attacker agent takes the fastest working route and has no obligation to follow the path a test author imagined, so advanced agent benchmarks need to verify both the result and how it was reached.

The benchmark has been tightened

Enclave says it has closed the unintended Grafana route and the Jenkins file-link shortcuts while keeping the planned weaknesses available under stricter path checks. The repaired challenges carry new source versions, and leaderboard comparisons will use matching benchmark versions, so models tested previously need fresh runs before ranking.

Why it matters

A model that can compromise realistic, widely deployed infrastructure end to end for the price of a coffee changes the economics of offensive security. Defenders now have to assume that capable autonomous attack tooling is rentable at near-zero marginal cost, not the preserve of well-funded teams. The result is also a caution for anyone evaluating AI agents: outcome-only scoring hid the fact that nearly half of DeepSeek's successful runs solved a different problem than the one designed, which is exactly the kind of blind spot that matters when the output is code execution on your servers.