Industrial document intelligence: cloud vs. on-premise OCR, LLMs and exception handling

Cloud vs. on-premise OCR: the main criterion is not price

The cloud / on-premise decision for OCR processing is structuring because it impacts the entire pipeline architecture. The determining criterion is not the cost per page, but the nature of the data being processed.

Case for the cloud (Azure Document Intelligence, AWS Textract, Google Document AI):

• Documents without sensitive technical data: generic supplier invoices, delivery notes, HR documents
• Variable volumes with peaks: the cloud absorbs peaks without over-sizing the infrastructure
• Need to start fast: cloud APIs are operational within a few days
• The organization has accepted processing this data on third-party servers (GDPR, security policy)

Case for on-premise:

• Documents containing proprietary technical data: chemical compositions, dimensioned drawings, formulations, manufacturing processes
• Regulatory or contractual data-localization requirements (defense, nuclear, aerospace)
• IT policy that prohibits processing production data on the public cloud
• Stable and high volumes making the cloud more expensive over 3 years

Azure Document Intelligence is available as a containerized on-premise deployment (Docker) with the same performance as the cloud version: it is often the compromise solution for sensitive documents.

LLM for extraction: prompt engineering or fine-tuning?

When prompt engineering is enough

Prompt engineering (without fine-tuning) works well on documents whose structure is relatively stable and whose fields to extract are clearly defined in the document. For material certificates in a standardized format (even if each supplier has its own layout), a well-designed prompt with a few few-shot examples reaches 80 to 88% accuracy.

Advantages: no training dataset, fast start, easy to modify. Drawbacks: edge cases (complex tables, unusual format, poor-quality scan) cause problems.

When fine-tuning is necessary

Fine-tuning on an annotated corpus becomes necessary when:

• The variability of formats is high (100+ suppliers with as many different layouts)
• You are targeting > 90% automation: prompt engineering plateaus below that
• The documents contain domain-specific technical terms poorly covered by the general-purpose model
• You have an annotated corpus of 500+ documents (below that, the benefit of fine-tuning is small)

Fine-tuning a 7B to 13B parameter model on 800 to 1,200 annotated examples generally yields 8 to 12 accuracy points more than prompt engineering alone on heterogeneous industrial corpora.

Validating accuracy on your corpus: the methodology

The accuracy claimed by OCR and LLM vendors is measured on generic benchmarks. On your specific corpus, results can be very different: in either direction.

The recommended validation approach:

• Build a representative test dataset: 100 to 200 documents covering the diversity of formats, suppliers, and scan qualities. Do not take only the good cases.
• Annotate the target fields manually on this dataset: this is the ground truth.
• Measure accuracy per field, not just overall. A field at 60% accuracy that goes into the ERP is unacceptable even if overall accuracy is 92%.
• Measure confidence score vs. actual accuracy: check that the model's confidence score is well correlated with actual accuracy. This is the condition for being able to use the confidence threshold as a routing criterion to human review.

Sizing human review: neither too much nor too little

Human review of exceptions is unavoidable: but its sizing directly determines the profitability of the project.

The confidence threshold above which a document is routed to human review is the key parameter. Too low: too many documents in review, the productivity gain collapses. Too high: errors slip into the ERP.

An effective approach: use a per-field confidence score rather than a global one. A document with 21 of 22 fields at high confidence and 1 uncertain field is more efficiently handled by partial review (just the uncertain field pre-highlighted) than by full review of the document.

The review interface must therefore be designed to minimize the operator's work: pre-filled fields, uncertain zone highlighted in the source document, validation shortcuts. An operator can process 60 to 80 documents / hour in well-designed partial review, vs. 20 to 30 in full review.

ERP integration: the points to watch

Integration into the ERP (SAP MM, Oracle, Sage) via API is more reliable than exporting CSV or Excel files. It enables real-time processing, per-record error handling and operation traceability.

Common points to watch:

• Duplicate handling: the same document can arrive through several channels (email + supplier portal). Deduplication before ERP integration is critical.
• ERP rejection handling: a record can be rejected by the ERP for business validation reasons (unknown supplier reference, unrecognized date format). These rejections must be caught and routed to a correction queue: not simply ignored.
• Non-regression tests: ERP updates can change validation rules and cause documents that used to pass to be rejected. Plan automated non-regression tests on a reference corpus.