DeepSeek V4.1 Flash 1017 beats September build in agentic coding benchmarks

A one-day-old point release outscored its predecessor

DeepSeek shipped V4.1 Flash 1017 on Saturday 17 October and described it as a minor update. By Sunday, a community benchmark published on dev.to had the new build ahead of the September V4.1 Flash release on both of the author's agentic coding tests.

On Gauntlet 7.2, where the open-source coding agent goose assembles a small payments app within 150 model calls and faces 99 automated checks, the 1017 build scored 0.6957 against 0.591 for September's release. On Forge 2.0, which hands the agent a working Jira Forge app plus a long specification for its second version, with 300 model calls and 105 checks across three seeded Jira sites, the new build posted a median of 0.4520 versus 0.3342. According to the post, a full day of running the model through OpenRouter cost at least 83 cents.

What changed, and what didn't

The post notes that DeepSeek's changelog leaves both the API name and the price alone, advising users that calling deepseek-flash now resolves to the newest model. Anyone pointing an agent at DeepSeek's own API may therefore have moved to 1017 over the weekend without making an explicit decision.

The 1017 model card claims broad gains in reasoning, coding, agentic and visual tasks, along with shorter outputs and faster execution, but leaves the architecture untouched. OpenRouter's listing repeats the September card's description of a 552-billion-parameter mixture-of-experts backbone that activates 8 billion parameters per token during prefill and 16 billion during decode.

Same task, same seeds, 0.24 and 0.66

The sharpest number in the post is the spread between two builds. The Forge 2.0 board lists each model as a pair of independent runs, and 1017's two builds, on the same task, starter app, seeds and scoring sites, finished at 0.2422 and 0.6618. The effort was almost identical, 262 tool calls in 28 minutes versus 270 in 30 minutes, and the raw work scores were fairly close at 0.8362 and 0.9693. The divergence came from the reliability multiplier, which penalises a build for failed checks across its 19 groups: build one fell short in 15 groups, build two in nine.

Behind several of build one's losses was a modal that never rendered. On one scoring site it opened twelve times and showed an empty frame every time, with no console errors or failed requests. The scorer's logs explained why: the app's resolver kept receiving a rate-limit response carrying a retry-after value of 2,237 seconds, and code carried over from the starter app did nothing but wait. Build one's worst group was UI function, at 0.0568 against build two's 0.9399.

Read the comparison with care

The author flags real limits on the headline comparison. Each figure comes from a single task per benchmark and, for Forge, a single build pair. The board also places the two Forge pairs in separate comparison groups because part of the harness changed between 12 and 18 October, and the September runs predate the app's switch to requesting medium reasoning effort, meaning they most likely ran at DeepSeek's default of high. Some of the gap, in other words, may belong to the benchmark rather than the model. Scores were read on the evening of 18 October, and the live run pages are the current record.

The same day of testing went worse elsewhere. NVIDIA's Nemotron 3.5 Super VL managed 0.3781 on the Gauntlet for $1.21, but its single Forge build fell to 0.0321 after a duplicated import prevented nine handlers from bundling. A locally run Muse Glimmer 30B reported the app ready even though all of its functions failed to build.

Why it matters

For developers choosing small, cheap models for agentic coding, the post carries two practical lessons. A point release labelled minor can move benchmark results substantially, and because DeepSeek keeps the API name stable, that change can reach production agents silently. Single-run scores are also a shaky basis for decisions: the same model on the same task produced a near-threefold spread depending on how one build reacted to a rate limit. Evaluations need multiple runs and a hard look at reliability, not just the headline work score.