Skip to content
Published on

OCR and Document Understanding Technical Reports: What to Read, and Why Parsing Is Not Finished

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
Split View필사 모드
Share
Authors

Introduction

Part two of the series is OCR and document understanding. This domain is oddly treated as a solved problem while it keeps hurting people in production. Anyone who has built a RAG pipeline has probably watched retrieval quality collapse not at the embedding stage but at the PDF parser.

Paper details were verified directly from the primary sources on 2026-08-12. This field moves fast, so check the current state yourself.

Why this article does not rank anything

Document parsing scores are especially treacherous. A benchmark keeps its name across versions while its document mix and scoring rules change underneath. How table structure recovery is normalized, and how strictly reading order is judged, moves the same model by several points.

So the reading here is not "who scored what" but which failure mode each report was aiming at. Every number quoted is an author-run measurement, not an independent evaluation.

What is actually hard in document understanding right now

[Where document parsing breaks]
 Layout  : multi-column, footnotes and captions scramble body order
 Tables  : structure recovery with merged cells and nested headers
 Math    : getting from an image back to markup
 Order   : reconstructing the order a human would read in
 Cost    : per-page cost is the ceiling on corpus size

The shared obsession of the recent reports is the last line. Cutting per-page cost by an order of magnitude changes what size dataset you can build, which matters more than nudging accuracy up a point.

Ten technical reports worth reading

OCR-free Document Understanding Transformer (Donut)

arxiv.org/abs/2111.15664 — submitted 2021-11-30, v5 on 2022-10-06. The referenced repository is clovaai.

The proposal is to remove the OCR engine from the pipeline entirely. The reasons the abstract gives still hold: the computational cost of OCR, inflexibility across languages and document types, and OCR errors propagating into every downstream step. The authors position this as a first step in OCR-free research.

Nougat: Neural Optical Understanding for Academic Documents

arxiv.org/abs/2308.13418 — submitted 2023-08-25.

A visual transformer that converts academic PDFs into markup. The framing in the abstract is the key part: PDF loses semantic information, and the loss is worst for mathematical expressions. If your corpus is math-heavy, understand this failure first.

General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

arxiv.org/abs/2409.01704 — submitted 2024-09-03.

The contribution is widening the definition of a "character". Plain text, but also mathematical and molecular formulas, tables, charts, sheet music and geometric shapes, all handled by one 580M-parameter model. Interactive region-level recognition guided by coordinates or color, dynamic resolution and multi-page input come along with it.

olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models

arxiv.org/abs/2502.18443 — submitted 2025-02-25, v3 on 2025-07-02.

An open-source toolkit built by fine-tuning a 7B vision language model on 260,000 pages drawn from over 100,000 PDFs. The number that matters here is not accuracy but cost: roughly 176 USD to convert a million pages, an order of magnitude away from commercial APIs by the authors' accounting. The accompanying olmOCR-Bench holds 1,400 PDFs.

DeepSeek-OCR: Contexts Optical Compression

arxiv.org/abs/2510.18234 — submitted 2025-10-21.

The problem framing is unusual. OCR is treated not as text extraction but as a context compression mechanism. The authors report 97% OCR accuracy below 10x compression and about 60% at 20x, and state that a single A100-40G can produce training data at over 200k pages per day. They explicitly call the work an initial investigation into feasibility.

DeepSeek-OCR 2: Visual Causal Flow

arxiv.org/abs/2601.20552 — submitted 2026-01-28.

The follow-up attacks token ordering. Their argument is that feeding visual tokens in a rigid raster-scan order with fixed positions conflicts with how human vision actually scans a page. DeepEncoder V2 reorders visual tokens semantically before the language model sees them, asking whether 2D image understanding can be handled by two cascaded 1D causal structures.

GLM-OCR Technical Report

arxiv.org/abs/2603.10910 — submitted 2026-03-11, v2 on 2026-03-16.

A 0.9B system pairing a 0.4B CogViT visual encoder with a 0.5B GLM decoder. The core moves are Multi-Token Prediction to accelerate decoding and a two-stage pipeline that runs layout analysis first and then recognizes regions in parallel. The authors target both edge deployment and large-scale batch production.

Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

arxiv.org/abs/2603.13398 — submitted 2026-03-11.

One 4B vision-language model covering parsing, layout analysis and comprehension. The interesting device is Layout-as-Thought: an optional thinking phase triggered by special tokens that produces a structured layout representation before the final output. The problem the authors name is worth taking seriously, that going end-to-end loses explicit layout grounding. Reported figures are 93.12 on OmniDocBench v1.5 and 79.8 on olmOCR-Bench, both author-run.

MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scale

arxiv.org/abs/2604.04771 — submitted 2026-04-06, v2 on 2026-04-09.

The observation in this report doubles as a diagnosis of the whole domain: architecturally different models fail on the same hard samples, which the authors read as evidence that data, not model design, is the binding constraint. They kept the 1.2B architecture fixed and grew training data from under 10M to 65.5M samples, used cross-model consensus for annotation reliability, and iteratively refined hard samples, reporting 95.69 on OmniDocBench v1.6. The earlier MinerU report (arxiv.org/abs/2409.18839) pairs well with it.

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

arxiv.org/abs/2607.04884 — submitted 2026-07-06, v2 on 2026-08-06.

This one takes lightweight-model latency head on. The authors report a 6.37x Transformer inference speedup and 2.14x under vLLM, while unifying document parsing, text spotting, information extraction, text-image translation and multi-image document understanding in one model. They also introduce Agentic Data Flow, an agent-driven system for building training data.

OCRBench v2

arxiv.org/abs/2501.00321 — submitted 2024-12-31, v2 on 2025-06-05.

The only non-model entry on this list, and possibly the one to read first. It spans 31 scenarios, 10,000 human-verified question-answer pairs, and a private test set of 1,500 annotated images. The authors' conclusion is unsentimental: most large multimodal models score below 50 out of 100, and they break down on less frequently encountered text recognition, fine-grained perception, layout perception, complex element parsing and logical reasoning.

Limitations the authors state themselves

  • The DeepSeek-OCR abstract explicitly calls the work an initial, feasibility-stage investigation. Reading its compression-ratio accuracy figures as product specifications is a mistake.
  • The Qianfan-OCR abstract concedes that end-to-end approaches lose explicit layout analysis capability, and presents its mechanism as a remedy for exactly that.
  • HunyuanOCR-1.5 names long-tail capabilities such as ancient-script OCR and low-resource multilingual parsing as areas still to improve.
  • MinerU2.5-Pro is a data-engineering result and does not claim to have addressed architectural constraints.
  • The low scores OCRBench v2 reports are the authors directly stating that this domain is not solved.

What a practitioner should read first

If you are building a pipeline for the first time, read OCRBench v2 first. Starting with knowledge of which failures remain is different from starting without it.

If you need to build a large corpus, olmOCR and MinerU2.5-Pro. The first gives you the per-page cost lens, the second the data quality lens.

If latency is the constraint, look at GLM-OCR and HunyuanOCR-1.5. Both treat decoding speed, not accuracy, as a first-class objective.

If context cost is what worries you, the DeepSeek-OCR line is genuinely interesting. Compressing text into images to save context is a different problem definition, not a better answer to the same one.

How and when this was verified

All ten reports here were verified on 2026-08-12 by opening the arXiv abstract page directly. Candidates that could not be opened were not cited. Every benchmark number is as reported by the authors, and changes with a different eval harness or prompt.

Try it yourself

  • OCR Text Extractor — OCR that runs directly in the browser. Feed it handwriting or a table and you will feel the failure modes these reports describe.
  • ML Training Data Explorer — see what shape of data document understanding models actually consume.

Previous in the series: Text LLM Technical Reports: What to Read

Next in the series: Video Generation and Understanding Technical Reports: What to Read

References

Discuss on Twitter • View on GitHub