Cloud / Google Cloud Document AI Interview questions

Document AI is a Google Cloud service that turns unstructured and semi-structured documents, such as scans, PDFs, and photos, into structured data. You send a file to a processor and get back text, layout, and extracted fields as JSON.

It is built on top of Vertex AI with generative AI, so newer processor versions use foundation models rather than only classic OCR and template models. Typical uses are invoice capture, KYC onboarding, claims intake, and feeding clean text into search or RAG systems.

flowchart LR
  A["Scanned file or PDF"] --> B[Processor]
  B --> C["Document JSON: text, layout, entities"]
  C --> D["BigQuery, app, or search index"]

Document AI groups processors by the job they do: digitize, extract, and classify or split. Choosing the right category is the first design decision in any pipeline.

Every processor also returns text and layout, so extraction processors include the digitize step internally.

A processor is the Document AI resource that does the actual work. It is created in a specific project and location, has a type such as OCR_PROCESSOR or FORM_PARSER_PROCESSOR, and exposes an endpoint you send documents to.

Its resource name looks like projects/{project}/locations/{location}/processors/{id}. A processor holds one or more processor versions, and requests go to the default version unless you target a specific one.

Think of it as a managed model endpoint that already knows how to read a certain kind of document.

Enterprise Document OCR digitizes documents. It detects and extracts printed and handwritten text from scans and PDFs in more than 200 languages, and returns pages, blocks, paragraphs, lines, and tokens with bounding boxes.

It also scores each page for readability, which helps you spot blurry or skewed uploads early. Optional add-ons cover math formulas, font style, and checkbox detection.

Use it when you need clean text and coordinates but do not need named fields. Downstream code, search indexing, or an LLM can then work with that text.

The Form Parser is a general-purpose extractor that needs no training. It detects key-value pairs, tables, and checkboxes in forms and returns them along with the OCR text.

It works well for surveys, applications, and questionnaires where the printed labels are the field names. The keys come from whatever is on the page, so you do not define a schema.

If you need specific, consistently named fields, such as policy_number, a Custom Extractor is the better fit.

The Layout Parser reads the structure of a document: text blocks, tables, lists, titles, headings, and page headers or footers. It accepts HTML and PDF input.

Its standout feature is context-aware chunking. It splits a document into chunks that follow the heading hierarchy, so all text in a chunk comes from the same layout entity such as a heading or list.

That makes it a common first step for retrieval-augmented generation and enterprise search, where cutting text at arbitrary character counts hurts answer quality.

A Custom Extractor pulls fields you define from documents Google has no pretrained parser for, such as a company-specific purchase order or a lab report.

You create a schema of labels, then pick an approach: a generative AI version built on a foundation model (usable zero-shot, with few-shot examples, or fine-tuned), a custom model trained on labeled documents, or a template-based version for fixed layouts.

Every version is evaluated on a test set before you deploy it, and deployed versions can be set as the default for the processor.

Specialized parsers are pretrained for a specific document type, so you can extract fields with no training. The documented set includes:

• Invoice Parser and Expense Parser for AP and receipts
• Bank Statement Parser, Pay Slip Parser, and W2 Parser for financial paperwork
• US Driver License, US Passport, and Identity Document Proofing parsers for identity checks
• Utility Parser for utility bills

They return typed entities such as invoice_id or total_amount with confidence scores. Availability differs by region, so check the processor list before designing around one.

A Custom Classifier looks at a document and decides which type it is, for example invoice, bank statement, or contract. The result comes back as an entity whose type is the predicted class, with a confidence score.

It is normally used as a router. Once a file is classified, your pipeline sends it to the matching extractor instead of running every extractor on every file.

You define the classes, supply labeled sample documents, train and evaluate a version, then deploy it. A generative AI foundation model can also back classifier versions.

A Custom Splitter finds where one document ends and the next begins inside a single multi-page file, such as a scanned stack of loan paperwork.

It does not cut the PDF for you. It returns entities with page references that mark each sub-document, and your code uses those page ranges to split the file.

A common pattern is splitter first, then classifier on each piece, then the right extractor per class.

Accuracy depends on how distinct the first page of each document looks, so train it on samples that include the awkward cases: cover sheets, blank separator pages, and documents that start mid-stack.

Document AI accepts common document and image formats: PDF, TIFF, GIF, JPEG, PNG, BMP, and WebP. Layout Parser additionally accepts HTML.

You must pass the correct mime_type with every request, for example application/pdf or image/png. A wrong or missing value is a frequent cause of INVALID_ARGUMENT errors.

Size also matters. Online requests are capped at 40 MB per file and batch requests at 1 GB, and each processor has its own page limits.

The Document is the JSON (or protobuf) message every processor returns. It is the single container for everything the processor found.

Positions in pages point back into text through text anchors, so you read text once and reference it by index.

An entity is one extracted or classified item, such as an invoice date, a supplier name, or a document class. It lives in document.entities.

The useful properties are type_ (the field name), mention_text (text as printed), normalized_value (typed value for dates, money, or addresses), confidence, and page_anchor for location.

Some entities are nested. A line_item entity, for instance, carries child properties for description, quantity, and amount.

Always check that a property exists before reading it. A missing optional field simply produces no entity, so code that assumes every key is present will fail on real traffic.

The confidence score is a value from 0 to 1 that tells you how sure the model is about an entity or a recognized element. Higher means more certain.

It is not a probability of being correct in a strict statistical sense, so treat it as a ranking signal and calibrate it against labeled samples from your own documents.

In practice you set thresholds per field. A document whose critical fields all clear the threshold flows straight through, and the rest go to a human review queue.

You can create one from the console or through the API. The console route is the quickest way to try things out.

1. Enable the Document AI API (documentai.googleapis.com) in your project.
2. Open Document AI and browse the processor gallery.
3. Pick a processor type, name it, and choose a location such as us or eu.
4. Copy the processor ID and use it in your process requests.

parent = client.common_location_path("my-project", "us")
processor = client.create_processor(
    parent=parent,
    processor=documentai.Processor(
        display_name="invoice-prod",
        type_="INVOICE_PROCESSOR",
    ),
)
print(processor.name)

Custom processors need extra steps afterward: dataset, labeling, training, and deployment.

Keep a separate processor ID for each environment, such as dev and prod, so test traffic never touches production versions or quotas.

Create a DocumentProcessorServiceClient pointed at the endpoint for your processor's location, build the processor name, wrap the file bytes in a RawDocument, and call process_document.

from google.api_core.client_options import ClientOptions
from google.cloud import documentai

client = documentai.DocumentProcessorServiceClient(
    client_options=ClientOptions(api_endpoint="us-documentai.googleapis.com"))
name = client.processor_path("my-project", "us", "PROCESSOR_ID")

with open("invoice.pdf", "rb") as f:
    raw = documentai.RawDocument(content=f.read(), mime_type="application/pdf")

result = client.process_document(
    request=documentai.ProcessRequest(name=name, raw_document=raw))
for e in result.document.entities:
    print(e.type_, e.mention_text, round(e.confidence, 2))

The endpoint prefix must match the processor location. A us processor called through the eu endpoint will fail.

For many files or very large ones, prefer batch processing instead, and wrap the call in retries for transient errors such as RESOURCE_EXHAUSTED. Reuse one client object across requests instead of creating a new one per file, and always read the document.error or exception details when a call fails, since they usually name the exact problem.

A processor version is a specific model release inside a processor. Pretrained versions have IDs like pretrained-layout-parser-v1.0-2024-06-03, while custom versions are ones you train or fine-tune.

A processor can hold several versions at once, and exactly one is the default. Calls to the processor use the default, and calls to the versioned resource name pin an exact version.

Versions have lifecycles. Older ones get deprecation dates, so production code should not assume a version lives forever.

Treat a version change like a code release. Evaluate it on a sample and then promote it, because even a minor model update can shift extracted values.

The Document AI Toolbox is an open-source Python library that wraps the raw Document message in friendlier objects. It saves you from hand-rolling code for anchors, shards, and exports.

Common helpers include loading sharded batch output straight from Cloud Storage, searching pages for a string, reading entities and tables as plain Python objects, and exporting results to BigQuery. It also has helpers for splitting a PDF using splitter output.

Use it to cut boilerplate, but keep the raw Document JSON in storage as the source of truth.

Document AI uses predefined roles that map to what a caller needs to do:

Give pipeline service accounts apiUser only, plus read and write access to the specific buckets. Keep editor and admin for people who build and train processors.

Storage permissions are separate from these roles. A batch job also needs read access to the input bucket and write access to the output bucket.

Online processing is a synchronous call: you send one document and wait for the Document in the response. Batch processing is asynchronous: you point at files in Cloud Storage, get a long-running operation back, and results are written to an output bucket.

Page limits per document are higher for batch and differ by processor, so check the limits page for the processor you use.

The Form Parser is generic and schema-less. A Custom Extractor is schema-driven and returns exactly the fields you defined.

A frequent path is to prototype with the Form Parser, then move to a Custom Extractor once downstream code needs fixed field names and types.

In practice many teams use both: online for the user waiting on a screen, and batch for everything that can wait.

It depends on how much labeled data you have and how varied your layouts are. Generative AI versions work from the first sample because the foundation model already understands documents. Custom model-based versions learn only from your labeled set.

A sensible default is to start with generative AI, measure on a test set, and fine-tune. Choose a custom model if you have lots of consistent, labeled documents and want predictable behavior.

Document AI stores all recognized text once in document.text. Every other element, whether a token, line, table cell, or entity, points into that string through a text_anchor.

A text anchor holds one or more text_segments, each with a start_index and end_index. To read an element's text you slice document.text between those indexes and join the segments.

def layout_text(layout, text):
    return "".join(
        text[int(s.start_index):int(s.end_index)]
        for s in layout.text_anchor.text_segments
    ).strip()

This design avoids duplicating text per element, and it means a single entity can cover several disjoint pieces of text.

Whichever you pick, evaluate both on the same test set so the comparison is fair, and look at latency and cost as well as F1.

Tables live under each page as page.tables. A table has header_rows and body_rows, each made of cells, and every cell has a layout whose text anchor points into document.text.

def cell_text(cell, text):
    return "".join(
        text[int(s.start_index):int(s.end_index)]
        for s in cell.layout.text_anchor.text_segments).strip()

doc = result.document
for page in doc.pages:
    for table in page.tables:
        header = [cell_text(c, doc.text) for c in table.header_rows[0].cells]
        rows = [[cell_text(c, doc.text) for c in r.cells] for r in table.body_rows]
        print(header, rows)

Guard against tables with no header row. Also remember that a table split across pages arrives as separate table objects that you may need to stitch together.

The same mechanism applies to entities, so mention_text is only a convenience copy of what the anchor points to. When an entity wraps across lines, expect several segments rather than one. That is why you should never search the final text for a value and guess its position. Follow the anchor and the geometry stays correct.

Build a BatchProcessRequest with an input location, either a prefix or an explicit list of files, and an output Cloud Storage path. The call returns a long-running operation you wait on or poll.

input_cfg = documentai.BatchDocumentsInputConfig(
    gcs_prefix=documentai.GcsPrefix(gcs_uri_prefix="gs://my-bucket/incoming/"))
output_cfg = documentai.DocumentOutputConfig(
    gcs_output_config=documentai.DocumentOutputConfig.GcsOutputConfig(
        gcs_uri="gs://my-bucket/results/"))

op = client.batch_process_documents(
    request=documentai.BatchProcessRequest(
        name=name, input_documents=input_cfg, document_output_config=output_cfg))
op.result(timeout=3600)

Make sure the identity running the job can read the input bucket and write the output bucket. When the operation finishes, read the metadata for per-file status, because one bad file does not fail the whole batch.

Use the Form Parser or a Custom Extractor with child labels when you need meaningful columns such as quantity or unit price. Raw OCR tables only give you positional cells.

Enterprise Document OCR has optional features you switch on through OcrConfig. Each one adds data to the output, and each can affect cost, so enable only what you read downstream.

• Image quality scores flag blurry, noisy, or skewed pages.
• Selection mark detection reports checkbox and radio states.
• Font style info returns bold, italic, and size, useful for spotting headings.
• Math OCR recognizes formulas as LaTeX.
• Native PDF parsing reads embedded text directly instead of OCRing it.

opts = documentai.ProcessOptions(ocr_config=documentai.OcrConfig(
    enable_native_pdf_parsing=True,
    enable_image_quality_scores=True,
    premium_features=documentai.OcrConfig.PremiumFeatures(
        enable_selection_mark_detection=True, compute_style_info=True)))

Pass opts as process_options on the request.

Use a prefix to process a whole folder, and an explicit file list when only certain files should run. Give each run a unique output prefix, such as a date or run ID, so results from different jobs never mix.

The schema is the list of labels the extractor must return. Each label has a name, a data type, and an occurrence.

• Data type: plain text, number, money, datetime, address, or checkbox. Typed labels populate normalized_value.
• Occurrence: required once, optional once, required multiple, or optional multiple.
• Parent and child labels: use a parent such as line_item with children for description, quantity, and amount to capture repeating rows.

Keep names snake_case and unambiguous. Avoid two labels that could both match the same text, such as date and invoice_date, because the labeling noise hurts accuracy. Start with the fields you actually use downstream and add more later.

Evaluation runs a version against a test set of labeled documents that were kept out of training. The result reports precision, recall, and F1, overall and per label.

Read the per-label numbers rather than only the headline F1. A strong overall score can hide a weak label such as tax_id. You can also move the confidence threshold to see how precision and recall trade off, and compare two versions side by side before promoting one.

Toggle fuzzy matching only when small formatting differences, such as spacing or punctuation, are acceptable in your downstream use. The same evaluation is available through the API with evaluate_processor_version.

Use Layout Parser when the structure of the document matters, not just its characters. It identifies headings, lists, tables, and paragraphs and can chunk by that hierarchy.

Use Enterprise Document OCR when you want faithful text and word-level geometry, such as for highlighting, redaction, or feeding your own post-processing.

Rerun the evaluation after every change to the schema or the test set, because scores from earlier runs are no longer comparable.

Layout Parser produces chunks that respect the document's structure, so each chunk comes from one layout entity such as a section or list. That keeps related sentences together, which improves what an embedding captures.

With include_ancestor_headings on, each chunk also carries the headings above it. A paragraph under Refund policy then still says it belongs to that section, even when retrieved alone.

opts = documentai.ProcessOptions(
    layout_config=documentai.ProcessOptions.LayoutConfig(
        chunking_config=documentai.ProcessOptions.LayoutConfig.ChunkingConfig(
            chunk_size=500, include_ancestor_headings=True)))

flowchart LR
  A["PDF or HTML"] --> B["Layout Parser"]
  B --> C["Chunks with headings"]
  C --> D["Embedding model"]
  D --> E["Vector index"]
  E --> F["Retrieval for the LLM"]

The two also combine well. Some teams use OCR for geometry-heavy tasks and Layout Parser for the structured copy used in retrieval. A quick test is to ask what the next step consumes: raw characters and boxes point to OCR, while sections and headings point to Layout Parser.

Set thresholds per field, based on the cost of a wrong value, and calibrate them on labeled samples from your own documents.

1. Run a representative labeled sample through the processor.
2. For each field, plot precision against confidence threshold.
3. Pick the lowest threshold that meets your error budget for that field.
4. Decide document pass or fail from the critical fields, not the average.

Track the straight-through rate and the error rate after launch, and revisit thresholds when document mix changes.

Use both when incoming files are a mix of unknown types and some are bundles. The splitter first cuts a scanned stack into logical documents by page range. The classifier then labels each piece, and the label picks the extractor.

If every upload is a single document of unknown type, you only need the classifier. If every upload is a bundle of known types, the splitter alone may be enough, because its output labels can already hint at the document type.

Add a fallback route for low-confidence classes, such as a manual queue, so a wrongly routed document does not produce confident but wrong fields.

Typed entities carry a normalized_value next to the printed mention_text. Use the normalized value for logic and keep the mention text for audit and display.

for e in doc.entities:
    nv = e.normalized_value
    if e.type_ == "total_amount" and nv.money_value.currency_code:
        m = nv.money_value
        print(m.currency_code, m.units + m.nanos / 1e9)
    elif e.type_ == "invoice_date" and nv.date_value.year:
        d = nv.date_value
        print(f"{d.year}-{d.month:02d}-{d.day:02d}")

Money is split into units and nanos, so combine them carefully, ideally with Decimal. Ambiguous dates like 03/04/2026 depend on the parser's reading, so validate them against business rules.

Measure each router stage separately. A splitter that cuts a page too early can look like a classifier failure later, which makes debugging confusing. Also keep the routing thresholds in configuration, not code, so you can tighten them without a redeploy when misroutes appear.

Pretrained processors serve the default version, and calling the versioned resource name pins a specific one. Custom versions follow a train, evaluate, deploy, then set-default flow.

1. Train or fine-tune a new version and evaluate it on the test set.
2. Deploy it and send staging traffic by calling its version name directly.
3. Compare outputs against the current default on real samples.
4. Set it as default only after it clears your acceptance bar.
5. Keep the previous version deployed until you are sure you can roll back.

Watch the deprecation list. Versions have shutdown dates, and a pinned version that retires will start to fail. Undeploy test versions you no longer need, since deployed custom versions can add hosting cost.

A processor is a regional resource. Documents are processed and data is stored in the location you pick, which matters for data residency and compliance.

The multi-region locations are us and eu. A handful of single regions, such as europe-west2 or asia-south1, have limited support and need an access request. Not every processor version exists in every location.

You cannot move a processor later. Changing location means recreating it and, for custom processors, re-importing the dataset and retraining. Call the matching endpoint, such as eu-documentai.googleapis.com, and keep Cloud Storage buckets in the same geography to avoid latency and egress surprises.

Document AI Human-in-the-Loop and Document AI Warehouse were deprecated on January 16, 2024, with shutdown scheduled for January 16, 2025. Human review is now something you build around the processor, or get from a partner.

1. Auto-approve documents whose critical fields clear their confidence thresholds.
2. Send the rest to a review queue, for example Pub/Sub plus Firestore.
3. Show the original page in a small review UI and draw boxes from each entity's page_anchor.
4. Store the corrected values with the reviewer and timestamp.

Feed the corrected documents back into your training or test dataset. That is how a custom processor improves over time on the layouts that actually fail.

Document AI bills mainly per page processed, and rates differ by processor, so cost control starts with sending fewer and cheaper pages.

• Use Enterprise Document OCR when text is enough, rather than a specialized parser.
• Select only needed pages with the page selector, for example just the first pages for classification.
• Turn on only the OCR add-ons you really use.
• Cache results in Cloud Storage keyed by a file hash so identical files are not reprocessed.
• Undeploy unused custom versions, since deployed versions can add hosting cost.
• Prefilter obvious junk, such as blank or unreadable pages, before sending.

Check the current pricing page for exact rates, since they change.

Enterprise Document OCR recognizes handwritten text as well as print, and it returns a quality assessment per page so you can see when input is poor. Handwriting accuracy still drops with messy writing, low resolution, and heavy skew.

Use the image quality scores to branch: pass good pages through, and send poor ones to a re-scan request or a review queue. Per-token confidence helps you target the specific words that are shaky.

Improve inputs when you can. Scan at a reasonable resolution, avoid shadows, and keep pages upright. Cleaning images upstream is usually cheaper than fixing wrong extractions later.

Measure cost per successfully processed document, not only per page, since reprocessing and review time add up.

A common design reacts to file uploads instead of polling. The flow is fully asynchronous, so a slow document never blocks the next one.

flowchart TD
  A["File lands in GCS inbox"] --> B["Eventarc trigger"]
  B --> C["Cloud Run service"]
  C --> D{Small file?}
  D -- Yes --> E["Online process_document"]
  D -- No --> F["Batch process with LRO"]
  E --> G["Save Document JSON to GCS"]
  F --> G
  G --> H["Publish processed message to Pub/Sub"]
  H --> I["Validation service"]
  I --> J{Critical fields pass?}
  J -- Yes --> K["Write to BigQuery or ERP"]
  J -- No --> L["Review queue"]

1. The upload fires a Cloud Storage event, and Eventarc invokes a Cloud Run service.
2. The service picks online or batch mode by file size and page count.
3. The raw Document JSON is written to Cloud Storage before anything else, so the result can be replayed.
4. A Pub/Sub message triggers validation, which either loads the data or opens a review task.

Make every step idempotent, for example by keying on the file hash, because Eventarc and Pub/Sub deliver at least once.

Add a dead-letter topic for failed messages so one poison file cannot block the queue, and alert on its size for early warning.

Large batch results are not always one file. Document AI can write a single processed document as several shards, each a Document JSON, under the output prefix grouped by operation.

Each shard has shard_info with a shard_index, a shard_count, and a text_offset. Those values let you sort shards, confirm none are missing, and stitch text and entities back together.

flowchart LR
  A[batch_process_documents] --> B["Long-running operation"]
  B --> C["Shards written to GCS output prefix"]
  C --> D["List and sort by shard_index"]
  D --> E["Check shard_count is complete"]
  E --> F["Merge text and entities"]
  F --> G["Downstream load"]

Do not assume the file name order is the page order. Use the Toolbox to load shards directly from Cloud Storage, and always verify the count before treating a document as complete.

When you merge shards, apply the text offset so anchors still point at the right characters. Also keep the shard list with the operation ID in your logs. If a read fails halfway, you can resume from the same set instead of rerunning the batch.

Start with evidence, not guesses. Open the evaluation, find the weak labels, and separate misses from wrong values.

Check the OCR output first, since extraction cannot beat bad text. Then look at schema ambiguity, label counts per field, and whether test and production documents match. Add post-processing rules, such as format checks, for fields that remain fragile.

When you exceed a quota, Document AI returns 429 RESOURCE_EXHAUSTED. Treat it, along with 503 UNAVAILABLE and deadline errors, as retryable using exponential backoff with jitter.

from google.api_core import exceptions, retry

r = retry.Retry(
    predicate=retry.if_exception_type(
        exceptions.ResourceExhausted,
        exceptions.ServiceUnavailable,
        exceptions.DeadlineExceeded),
    initial=1.0, maximum=60.0, multiplier=2.0, timeout=300)

result = client.process_document(request=req, retry=r, timeout=120)

Retries alone do not fix sustained overload. Limit concurrency with Cloud Tasks or Pub/Sub flow control, move bulk work to batch processing, and request a quota increase when your steady load is legitimately higher. Do not retry INVALID_ARGUMENT, since the same request will fail again.

Log the request ID and error code on every retry so you can tell a short blip from a quota that needs raising, and cap the attempts so a failing document does not retry forever. Alerts on the rate of 429 responses give you an early signal that traffic has outgrown the current quota before users notice delays.

The aim is to accept any upload and still run only the extractor that fits. Use the splitter and classifier as routers, and keep a human fallback for anything uncertain.

flowchart TD
  A[Upload] --> B["Custom Splitter"]
  B --> C["Sub-documents by page range"]
  C --> D["Custom Classifier"]
  D --> E{Class and confidence}
  E -- Invoice --> F["Invoice Parser"]
  E -- Bank statement --> G["Bank Statement Parser"]
  E -- Contract --> H["Custom Extractor"]
  E -- Low confidence --> I["Manual queue"]
  F --> J[Validation]
  G --> J
  H --> J

Store the original file, the split ranges, the class, and the extractor result together under one case ID so any step can be audited. Version the routing table in code, so adding a new document type is one extractor plus one route, not a rewrite.

Test each route with a deliberately misclassified sample. Wrong classification is the hardest failure to notice, because the wrong extractor still returns plausible-looking fields. Keep the routing table small and explicit, with one owner per route, so a new document type does not quietly change how an existing one is handled.

Security is layered, since documents often hold personal and financial data. Treat the extracted JSON as sensitive as the source file.

Review Google's data processing terms for your use. Also remember that review UIs and logs can leak data, so never log full entity values for sensitive fields.

Run periodic access reviews on service accounts and buckets, since pipelines tend to collect extra permissions over time.

Fine-tuning adapts a foundation model version to your layouts and labeling conventions. It usually follows a zero-shot baseline, so you know what the tuning actually bought you.

1. Define the schema and import a representative document set.
2. Auto-label with the foundation model, then correct the labels by hand.
3. Split into training and test sets, keeping the test set untouched.
4. Fine-tune to produce a new version of type Generative AI.
5. Evaluate it against the zero-shot baseline, per label.
6. Deploy and set as default only if it wins on the labels you care about.

Plan for upkeep. Base foundation versions have deprecation dates, so keep your labeled dataset and be ready to re-tune on a newer base when needed.

Keep a small frozen validation set that is never used while labeling, so later version comparisons stay honest.

Flatten each Document into one row per document, with repeating items such as line items in an ARRAY<STRUCT> column. Keep the raw JSON in Cloud Storage and store its URI in the row.

rows = [{
    "doc_id": file_hash,
    "gcs_uri": json_uri,
    "processor_version": version_name,
    "invoice_id": get("invoice_id"),
    "total_amount": float(total),
    "currency": cur,
    "line_items": [{"description": d, "amount": float(a)} for d, a in items],
}]
bq.load_table_from_json(rows, "proj.ds.invoices").result()

Use load jobs rather than streaming for large volumes, and partition by processing date. The Toolbox also has an entities_to_bigquery helper for simple cases. Record the processor version so you can explain a value later or reprocess one version's output.

Keep a second table for low-confidence or failed documents so analysts can see data quality without opening raw JSON. For dashboards, expose a view that unnests line items and joins the confidence columns, so analysts do not need to know the storage layout. Partition by processing date and cluster by supplier to keep those queries cheap.

Usually it is a coordinate mismatch, not a wrong extraction. Document AI gives positions as normalized_vertices, values from 0 to 1 relative to page width and height, and also as pixel vertices.

Common causes:

• Multiplying normalized values by the wrong image size, such as a thumbnail instead of the page dimensions.
• Drawing on a rotated or deskewed page while using the original orientation.
• Ignoring PDF page boxes or user units when rendering with a different library.
• Using only the first page reference for an entity that spans pages.

Fix it by reading page.dimension and any transforms, scaling normalized values by your rendered pixel size, and drawing per page_ref. Test with a rotated sample page, because that case breaks most overlays.

Each processor has its own page limits, and online mode is stricter than batch. If a file is over the limit, process it in parts and merge the results.

1. Prefer batch mode first, since its page limits are higher.
2. If still over, split the file into page ranges with a PDF library, or use the page selector in ProcessOptions.
3. Process parts in parallel and keep each part's starting page.
4. Add the offset to page numbers and text indexes when merging.
5. Keep logical documents together, for example a table that crosses a split point.

from pypdf import PdfReader, PdfWriter

def parts(path, size):
    r = PdfReader(path)
    for start in range(0, len(r.pages), size):
        w = PdfWriter()
        for i in range(start, min(start + size, len(r.pages))):
            w.add_page(r.pages[i])
        yield start, w

Overlap parts by a page or two if a heading or table may straddle the boundary, then deduplicate the overlap.

Test the merge logic with a table that crosses a split point, because that is where most bugs show up.

Use each for what it does best. Document AI gives reliable text, layout, and typed fields with confidence and locations. An LLM, such as Gemini on Vertex AI, is better at summarizing, reasoning across sections, and answering open questions.

• Fields first: take amounts, dates, and IDs from Document AI entities, not from the LLM.
• Grounded prompts: pass Layout Parser chunks, with headings, as context for questions and summaries.
• Verification: check any quoted value from the LLM against the document text through text anchors.
• Escalation: send disagreements between the two to human review.

This keeps numbers auditable and limits hallucination. It also makes cost easier to control, since the LLM only sees the chunks it needs.

Log both the Document AI value and the LLM output whenever they disagree. That log becomes a ready source of hard examples for later tuning.

Never post straight from the model. Run business checks that catch both extraction errors and genuine invoice problems.

flowchart LR
  A["Parsed invoice"] --> B["Field and confidence checks"]
  B --> C["Totals reconciliation"]
  C --> D["Duplicate and vendor checks"]
  D --> E{All pass?}
  E -- Yes --> F["Post to ERP"]
  E -- No --> G["Review queue with reasons"]

Compare money with a small tolerance for rounding, and log which rule fired so reviewers can fix the right field fast.

Store the rule results with the invoice so auditors can see why it was approved or held, and tune the rounding tolerance per currency, since some currencies have no minor units.