Qwen3 4B matches Qwen2.5 7B on writing correction at half the cold-start time

A hands-on benchmark published on dev.to ran three locally hosted Qwen models through the same writing-correction workload and found that the smaller, newer Qwen3 4B kept pace with the larger Qwen2.5 7B on every single case, while responding in under half the cold-start time on the tested Windows setup.

How the test was run

The author, posting as linguapilot-ai, used 20 paired writing cases: 16 sentences containing expected errors and 4 controls that were already correct. Each case went through three models hosted locally with Ollama on Windows — qwen2.5:7b plus quantized q4_K_M builds of qwen3:4b and qwen3:8b — for a total of 60 local responses, all judged against an unchanged reference using one evaluation framework.

The framework scored several dimensions independently: whether expected errors were fully corrected, correction coverage, localization of error regions, preservation of the original facts, adherence to the requested output structure, explanation-language behavior, and cold-start execution time. A case only counted as complete when every expected fix was applied without a wrong replacement, a false error, or configured fact loss.

The experiment follows an earlier round that compared Qwen3 4B, 8B and 14B, in which the two larger models both reached 19 of 20 complete cases.

Correction results

On the primary metric, the two headline models were indistinguishable. Qwen3 4B and Qwen2.5 7B each completed 18 of 20 cases (90%) — and, more strikingly, they succeeded on exactly the same 18 cases and failed on the same 2. Not a single paired case produced a different outcome between them.

Qwen3 8B edged ahead at 19/20 (95%), and that lead carried into secondary measures. It localized all 21 expected error regions for a 100% F1 with no false positives, while Qwen3 4B and Qwen2.5 7B each localized 20 of 21 (97.6% F1). Expected-correction coverage followed the same pattern: 19/21 (90.5%) for both the 4B and 7B models against 20/21 (95.2%) for the 8B.

Speed and explanation behavior

Timing is where the gap widened. In the tested cold-start configuration, Qwen3 4B averaged 23.99 seconds per response, versus 54.37 seconds for Qwen2.5 7B and 60.24 seconds for Qwen3 8B. That makes the 4B model roughly 2.27 times faster than the 7B — about a 56% reduction — while the 7B and 8B sat far closer together, with Qwen2.5 7B about 9.7% quicker than Qwen3 8B.

Explanation quality produced the one clear difference favoring the older model. According to the benchmark, Qwen2.5 7B passed all 40 evaluated explanation-language fields and Qwen3 8B passed all 41, whereas Qwen3 4B recorded 37 passes with 3 warnings. All three still emitted parseable JSON, met the output contract in every case, passed the fact-preservation check on all 20 cases, and produced consistent action-explanation pairs on all 16 error-containing sentences.

Why it matters

For anyone building a local writing assistant, the result reframes model choice as a trade-off across several criteria rather than a simple contest of size. A newer 4B model delivered the same correction outcome as a previous-generation 7B model on every case, at roughly half the measured cold-start latency — a meaningful margin on consumer hardware where memory and response time are the binding constraints.

The choice is not purely about speed. Qwen2.5 7B showed cleaner explanation-language behavior, and Qwen3 8B bought one extra complete case, one extra expected correction and full error localization at the price of the slowest cold-start time of the three. An application that prizes latency, coverage or explanation discipline could reasonably crown a different winner.

Caveats

The author is explicit about scope. Twenty paired cases is a focused set, too small to support a universal ranking, and the timings describe one configuration — Ollama on Windows, cold-start runs — rather than a general speed ordering. The task is also narrow: correcting English sentences with explanations requested in French, so other languages and task types may behave differently. The automatic checks capture output discipline and consistency, not deeper pedagogical quality, which would require additional linguistic or human evaluation.