Ingestion, not model choice: why PDF parsing decides RAG retrieval quality

A dev.to article argues that many RAG failures blamed on the model are actually ingestion failures. If a parser flattens a PDF into a character stream and a splitter cuts that stream at fixed intervals, even the best embedding model and the best LLM end up working on fragments rather than meaning.

Flat extraction loses what retrieval needs

A standard PDF extractor returns a character stream, and the article lists three casualties: the reading order of multi-column pages, the link between a heading and the paragraphs beneath it, and the row-and-column relationships inside tables. LangChain's recommended default for generic text, RecursiveCharacterTextSplitter, keeps paragraphs intact where it can and falls back to sentences, then words. That is a sound baseline for clean prose, but as the author notes, it only sees separators — it cannot see a table.

To demonstrate, the author split a mock 10-K filing with a deliberately small 200-character budget. The output was three broken chunks: a revenue table severed from its heading and from the sentence explaining the numbers, a Risk Factors heading stranded at the end of the table chunk, and a lone risk sentence with no section attached. Larger budgets, like the 1,000-character size with 200 overlap used in LangChain tutorials, only delay the failure; any fixed budget eventually lands inside a table or between a heading and its body. Retrieval quality, the article argues, is decided at this step, before a single embedding is computed.

Document intelligence comes before chunking

The proposed fix is to read the page before cutting it. Document intelligence does four jobs: layout analysis that detects and classifies regions such as titles, section headers, body text, tables and figures; reading order; table structure recognition; and OCR for scanned pages. The Docling technical report, cited in the article, describes an object-detection layout model that predicts bounding boxes and element classes, plus a second model, TableFormer, which recovers table structure — including partial or missing borderlines, empty cells and spans — before assembling everything into one document object that exports to Markdown or JSON.

Training data matters here too. According to the article, DocLayNet was created because earlier public datasets came only from scientific article repositories, and models trained on them degraded on other page designs. It contains 80,863 manually annotated pages with bounding boxes in 11 classes, drawn from finance, science, patents, tenders, law texts and manuals, and Docling's layout model was trained on it among other datasets.

A toolbox with different trade-offs

The article compares four options:

• Docling: IBM Research's MIT-licensed conversion toolkit. Recovers layout, reading order, figures and tables via TableFormer, runs locally on commodity hardware and has a LangChain loader.
• PP-DocLayoutV2: PaddleOCR's layout stage, an RT-DETR-based detector paired with a pointer network, covering 25 element categories and reading order. PaddleOCR's documentation already lists a newer V3.
• Mistral OCR 3: a hosted API the article dates to December 17, 2025, emitting Markdown with HTML tables that preserve merged cells. Its claimed 74% win rate over the previous version is vendor-reported.
• Infinity-Parser: a research vision-language parser trained with LayoutRL that produces structured output directly from page images, with a reward combining edit distance, paragraph count and reading order.

The selection advice is blunt: test tables first, because Markdown cannot express merged cells while HTML can, and push the worst pages you have through each candidate before committing.

Structure-aware chunks in LangChain

Integration comes through DoclingLoader, which shares LangChain's standard loader interface. ExportType.MARKDOWN returns one document per file, while ExportType.DOC_CHUNKS — the default — returns one document per chunk from Docling's own chunker, with HybridChunker as the default. The walkthrough was run with langchain 1.4.7, langchain-docling 3.0.0, docling-slim 2.145.0 and langchain-text-splitters 1.1.3 on Python 3.13.

The article's closing rules: parse for structure first, because layout, reading order and table recovery decide what chunks can possibly contain; chunk along that structure, with sections, tables and parent-child links leading and token limits as a backstop; keep headings and page numbers as metadata so retrieval can filter and agents can cite; and for visually dense pages, skip text extraction and retrieve over page images instead.

Why it matters

The article inverts the usual debugging order: ingestion, not model choice, decides retrieval quality. It cites the authors of ColPali, a visual retrieval model, who found in their experiments that for visually rich documents an improved ingestion pipeline helped retrieval far more than swapping the text embedding model. The practical consequence is that inspecting what your parser and splitter actually produce, using your own worst documents as the test set, is cheaper than upgrading embeddings or LLMs to compensate. And for agents that call retrieval as a tool, structural metadata pays twice: it enables filtered lookups and gives the model something it can cite.