Book a 30-min call
← cd ../blogs
$ cat posts/document-ai-sub-1b-specialist-beats-frontier.mdx

Document AI: when a sub-1B specialist beats a frontier model

May 14, 2026 · ImmovableTech Team

  • Document AI
  • Computer Vision
  • Inference
  • Production AI

The model that wins the benchmark is not the model that ships

For bounded document extraction, a sub-1-billion-parameter specialist beats a frontier vision-language model on every metric that decides whether the pipeline goes live. Not on general capability — on cost per page, on p99 latency, on whether you can run it inside the customer’s network, and on whether you can fine-tune it on their actual forms.

The evidence is no longer arguable. PaddleOCR-VL-1.5, released on 29 January 2026, is a 0.9B model — a NaViT-style native-resolution vision encoder on an ERNIE-4.5-0.3B language backbone — and it scores 94.50% overall on OmniDocBench v1.5. In the same table the authors put Gemini-3 Pro at 90.33% and Qwen3-VL-235B-A22B-Instruct at 89.15%. Those two rows were measured by the PaddleOCR team rather than taken from the official leaderboard, which is worth holding in mind, but the gap is four to five points and it runs the wrong way for a model roughly 250 times larger.

That is the comfortable half of the argument. The uncomfortable half is that “small specialist” is not a category you can buy safely, and the same benchmark family will show you why.

The cost arithmetic, done properly

Start with throughput you can check rather than a vendor claim. The PaddleOCR-VL-1.5 paper reports end-to-end inference across seven hardware configurations at 72 DPI. On a single A10 with the FastDeploy backend it does 1.15 pages per second in 21.8 GB of VRAM — call it 4,130 pages an hour. The nearest thing you can rent is a g5.xlarge, which carries a 24 GB A10G rather than an A10 and lists at $1.006 an hour on-demand in us-east-1, so treat what follows as the right order of magnitude rather than a quote.

Divide one by the other and you get roughly $0.24 per 1,000 pages at full utilisation. Amazon Textract’s Detect Document Text API is $1.50 per 1,000 pages for the first million pages a month. Textract’s Analyze Document Forms feature — the one that actually returns key-value pairs, which is what anyone doing claim forms needs — is $50 per 1,000 pages at the same tier.

Two hundred times cheaper than Textract Forms is not the useful number, because nobody runs a GPU at 100% duty cycle. The useful number is the break-even utilisation, and it is different for each comparison:

  • against plain OCR at $1.50 per 1,000 pages, you need about 16% utilisation, which is roughly 16,000 pages a day;
  • against the over-a-million-pages tier at $0.60 per 1,000, you need about 41%, or roughly 40,000 pages a day;
  • against structured form extraction at $50 per 1,000 pages, you need about 0.5% — roughly 500 pages a day.

That spread is the whole decision. If the customer wants markdown text out of a modest volume, self-hosting is a rounding error dressed up as a project, and the arithmetic says use the API. If they want structured fields out of a recurring form type, the specialist pays for itself at a volume most back-office teams clear before lunch. And none of the above prices the engineering: a GPU you operate is a thing on-call carries, and the first year of that costs more than the inference does.

Latency, and where the p99 actually comes from

Median latency is not where a frontier VLM loses; the tail is. A hosted model gives you someone else’s queue, someone else’s rate limits and someone else’s incident, and you find out about all three at the same time your customer does. A 0.9B model in 22 GB of VRAM gives you a p99 you own, and the lever you pull is capacity rather than a support ticket.

The lever that matters most is batching, and it is easy to get wrong in the direction of latency. PaddleOCR-VL-1.5’s own serving design runs input preparation, layout analysis and VLM inference as separate threads with queues between them, forming mini-batches either when the queue fills or when the oldest item has waited past a deadline. That deadline is your p99 knob, and we have watched a team tune it purely for throughput and then be surprised that single-page interactive uploads inherited the batch timeout. Batch the bulk path, run the interactive path unbatched, accept the lower GPU efficiency.

The other tail we did not anticipate: pages are not uniform. A dense multi-column journal page emits several times the visual tokens of a one-field remittance advice, so per-page latency has a fat right tail before you have added any concurrency at all. Size the queue on tokens, not on pages.

Deploying and fine-tuning are the same argument

The two strongest reasons to run a specialist have nothing to do with accuracy.

The first is that it goes where the documents are. Document AI work concentrates in insurance, healthcare and banking, which is to say in organisations whose security review will not approve posting claim images to a third-party endpoint. Open weights under Apache 2.0 — which is PaddleOCR-VL-1.5’s licence — turn a six-month procurement conversation into a container. The CPU and edge path matured this year too: llama.cpp merged DeepSeek-OCR support on 25 March 2026, shipping in release b8530, with GGUF weights published under ggml-org/DeepSeek-OCR-GGUF; GLM-OCR landed a little earlier, on 18 February 2026.

# Serve a document model with no GPU and no egress
llama-server -hf ggml-org/GLM-OCR-GGUF
# then POST an image_url to /chat/completion with the prompt "OCR"

The GLM-OCR pull request carries a warning that generalises to the whole class, and it is the sort of thing that only shows up once you have deployed one: these models must be driven through the API with the specific prompt they were trained on, and going through a general chat UI degrades output. The failure looks like a model problem and is a prompt problem.

The second reason is fine-tuning, and this is where the specialist stops being a cheaper frontier model and becomes a different product. PaddleOCR’s own SFT documentation walks through supervised fine-tuning of the 0.9B model on a single GPU with erniekit train, using a Bengali recognition dataset as the worked example. Substitute one insurer’s claim forms for the Bengali corpus and you have a model that has seen their layouts, their abbreviations and their handwriting conventions. No frontier API offers you that, and no amount of prompt engineering substitutes for it.

Where the specialist falls over

Now the part that gets teams into trouble. Being small does not make a model brittle, but being trained on a narrow distribution does, and the two are correlated in practice because small specialists are usually trained narrowly.

Real5-OmniDocBench, published alongside PaddleOCR-VL-1.5, takes the OmniDocBench v1.5 pages and physically reconstructs them under five conditions — scanning, warping, screen photography, illumination and skew. It exists precisely because clean-scan benchmarks measure the wrong thing. Look at what it does to Dolphin-1.5, a 0.3B document specialist: 83.39 on scanned pages, and 28.16 on skewed ones. Same model, same task, a 55-point collapse from a distortion any phone camera introduces by accident.

Then look at Gemini-3 Pro across the same five conditions: 89.47, 88.90, 88.86, 89.53, 89.45. A spread of two-thirds of a point. It never wins a category and it never falls over, which is exactly the behaviour you want from a fallback.

PaddleOCR-VL-1.5 lands at 92.05 overall with a range of 91.25 to 93.43, so a sub-1B model can be robust — but only because its authors deliberately trained and evaluated for these distortions. Robustness was engineered in, not inherited from the architecture. When you pick a specialist off a leaderboard you are picking its training distribution, and almost nobody publishes what that distribution was.

So the real engineering question is not “which model is better”. It is what shape your document distribution has. Narrow and high-volume — one insurer, four form types, 40,000 pages a day — favours the specialist overwhelmingly. Long-tail and unpredictable — an inbox that receives whatever a broker chose to send — favours the frontier model, because graceful degradation is worth more than four points of headline accuracy. Most teams pick wrong because they benchmark on a clean sample rather than on their tail.

What OmniDocBench does and does not measure

OmniDocBench v1.5 scores a page as one number derived from three: text edit distance, table TEDS and formula CDM. Reading order is reported separately. That composite tells you whether a model transcribed a page. It does not tell you whether the fields you care about came out right, and those are not the same question — a claim form can score well on character accuracy while putting the policy number in the wrong row.

The benchmark maintainers know the harness itself is a moving target. The 10 April 2026 v1.6 update added Multi-Granularity Adaptive Matching, which searches for the best segmentation granularity on the prediction side so that a model is not penalised for chunking a page differently from the annotator, and added 296 deliberately harder pages covering nested tables, dense formula layouts and unconventional layouts. Scores moved because the matching changed, not because the models did. Any comparison you inherited from before that date is measuring something slightly different from one taken after it.

Build your own set. A few hundred pages sampled from the customer’s real intake, labelled at the field level, weighted towards whatever arrives crumpled. It is a dull fortnight of work and it is the only number that predicts production. We build the eval set before the pipeline now for the same reason we run new models in shadow mode against the incumbent before they decide anything, which is a habit we picked up building fraud scoring and have never regretted.

What we’d do differently

We would stop treating this as a model-selection problem and route from the start. The right architecture is a specialist doing extraction, a confidence and image-quality gate in front of it, and a frontier model as the fallback for pages the gate rejects — the same shape as the verification pipeline we built for text. We arrived at that after shipping specialist-only and discovering the tail in production, which is the expensive order to learn it in.

The specific failure was ours and it was avoidable. Our holdout was drawn from the customer’s document management system, so every page in it was a clean 300 DPI scan. Real intake included phone photographs taken at a desk. On those, character accuracy stayed high enough that our text-level checks passed while reading order scrambled on two-column pages, so the extractor confidently associated values with the wrong labels. Silent, plausible, wrong — the worst combination. We spent three weeks adding a quality gate and a fallback route, and p99 latency got worse before the error rate got better, because the gate added a preprocessing pass to every page to protect maybe 4% of them.

The other thing we would change: measure the break-even utilisation before proposing self-hosting, not after. On one bounded pilot the honest answer was that the volume did not justify a GPU, and we would have reached it faster with a spreadsheet than with a benchmark.

References

We build and deploy document extraction pipelines as part of our AI & Machine Learning Engineering practice. Talk to us if you are deciding between a hosted document API and a model you run yourself.