Apple Vision OCR runs 300x faster than a local 27B VLM but destroys table structure

A developer comparing OCR options for local use found that Apple's built-in Vision framework extracted text roughly 300 times faster than a locally hosted 27B vision-language model — and still lost the comparison, because its output silently destroyed the structure of the document it had just read.

The setup

According to a hands-on benchmark posted on dev.to, the author ran a 27B vision model locally, using an IQ4_XS quantization that kept about 15GB resident, against macOS's Vision framework and its VNRecognizeTextRequest API — a dedicated text-recognition engine that is free and adds no memory footprint. The expectation going in was conventional: the specialist should win on character accuracy, and the general-purpose model should be reserved for semantic work.

Instead of testing on real documents, where no ground truth exists to separate plausible output from correct output, the author rendered a 1100×720 test image with the answer fixed in advance: a title and date in Korean, a four-column by three-row table (model, memory, speed, status), four lines of prose, two empty table cells containing hyphens, and one adversarial line — A0-1lO9-B8 — pairing digit 1 with lowercase l and capital O with digit 0.

The first render had to be thrown out: the Korean label came through as tofu boxes because the monospace font lacked the glyphs. As the author notes, if the ground truth is broken at the moment you fix it, every measurement afterwards is void — and the check costs thirty seconds.

The results

Metric	Local 27B VLM	Apple Vision
Character errors	2	8
Reading order	Preserved	Destroyed
Table cells dropped	0	2
Wall clock	82.8s (cold)	0.27s

Vision's eight errors included reading IQ4_XS as I04_XS, cloud as cLoud twice, tok/s as tok/5, two characters inside a code string, an em dash flattened to a hyphen, and an arrow rendered as ->.

Structure, not spelling, decided it

Vision returned the table decomposed by column: all three model names first, then the memory figures, with the speed and status columns appended at the end of the document. The characters were correct and well-formed, but the row associations were gone — not garbled, simply absent, and unrecoverable from the output. The VLM kept the rows intact. Next to that, two errors versus eight is, in the author's phrasing, a rounding difference.

The post frames the lesson narrowly: whether a dedicated tool beats a general one depends entirely on where you cut the task. Vision is dedicated to character recognition, not document understanding, and the benchmark was selecting tools by the first criterion while the job required the second.

Why cheap-first escalation breaks

The obvious architecture — run the cheap engine, detect failure, escalate to the expensive one — requires failure to be detectable. Column-shredded output is syntactically perfect, with plausible text and no error signal, so downstream code cannot distinguish it from a correct read. Dropped information leaves no residue. The author generalizes the point beyond OCR: any escalation ladder, whether cheap model to expensive model or cache to origin, is only sound when the cheap tier's failure mode is observable at the boundary. Otherwise it is an undetected data-loss path with a cost saving attached.

Homoglyphs beat both engines

On the adversarial string, the VLM produced A0-1109-B8 from the full image; given a 4x enlarged crop and an explicit instruction to distinguish 0/O and 1/l, it recovered the l but still lost the O (A0-1l09-B8). Vision on an enlarged crop was unchanged. The author inspected the enlarged image and found that the font renders zero with a slash and capital O as a plain oval — visibly different glyphs, meaning both engines genuinely misread a distinguishable character and more resolution does not help. The conclusion: no OCR engine should be trusted on strings where homoglyphs change the meaning — codes, IDs, hashes, addresses, license keys — and the remedy is human confirmation or a checksum, not a better engine.

What the author adopted

Vision as the first pass, escalating to the VLM only for documents where reading order carries meaning, such as tables and forms — with one correction: because column-shredding is not detectable downstream, table-heavy corpora should go to the VLM first and absorb the 300x cost. The author also lists what would change the verdict: prose-heavy documents make Vision alone sufficient, and if Vision gains layout analysis and starts preserving table structure, the conclusion is dead. The homoglyph failure is shared by both engines, so improving either does not touch that part.

Why it matters

For anyone running models locally, the benchmark is a useful corrective to tool selection by headline metric. A free engine that finishes in 0.27 seconds looks like the obvious default until it silently discards the relationships that made the document worth reading — and the failure is invisible to downstream validation. Speed and even character accuracy are the wrong axes when the task is document understanding, and any cheap-first pipeline needs to ask whether its cheap tier can fail by omission rather than by error.