Why We Chose Databricks ai_parse_document For A Production RAG System
By Romulo Silva, AI Engineer at Factored
If you have ever tried to improve a RAG system, you probably started where everyone starts: retrieval.
You increased top_k. You added reranking. You widened the candidate set. You tuned prompts, adjusted chunk sizes, changed metadata filters, and tried to make the generator cite more carefully. For a while, each change helped.
Then the gains flattened, and the failures started to look strange. The right document was often already in the retrieved set, but the answer was still wrong.
That changed the diagnosis. The bottleneck was no longer whether retrieval could find the source. It was whether the parser had preserved the evidence retrieval needed.
That is why we compared Databricks ai_parse_document with Azure Content Understanding (CU), for a RAG system we operate over a large slide-heavy document corpus. We wanted to answer one practical question: when the RAG stack is held constant, which top-tier parsing technology produces chunks that are easier to retrieve, ground, and trust?
We chose ai_parse_document because it surfaced the correct document about 40% more often, grounded numeric values about 15% more often, and cost about a third less than Azure Content Understanding's cheapest configuration.
1. Problem: The Parser Sets the Ceiling
Retrieval Was No Longer the Bottleneck
Think of the pipeline as a research assistant. Retrieval is the assistant finding the right report on the shelf. Extraction is the assistant reading it. If the assistant can find any report in seconds but cannot read a table, you do not have a search problem. You have a document-understanding problem.
In practice, that problem shows up in three recurring ways:
- The confidently wrong number: the system retrieves the right document, cites the right slide, and still returns a number from the wrong row.
- The invisible slide: a deck communicates through a chart with little or no text layer, so the page looks obvious to a human and nearly empty to a text-first parser.
- The plateau: every retrieval tweak helps less than the one before, because the missing signal was never in the chunks.
The most painful version was the confidently wrong number. Imagine a slide with this structured content:
A weak parse can collapse that into:
Segment Share Youngest cohort 18% Price-driven buyers 34%
The values survived, but the relationships did not. Ask the model for the youngest cohort, and 34% becomes a plausible answer. Retrieval did its job. The parser destroyed the answer before retrieval started.
That is the kind of failure that makes a RAG system hard to operate: the citation looks right, but the evidence behind it is no longer trustworthy.
Slide decks make this harder. The answer often lives in a table, chart, image-only slide, caption, or page-level relationship. If parsing flattens that structure, no retriever can fully recover it later.
The parser sets the ceiling for every downstream RAG step.
2. Candidates: Azure vs Databricks
Why These Two Technologies
This was not a comparison against a strawman. Azure was the serious incumbent path: it was the platform the ingestion pipeline already ran on, and Content Understanding represented Azure's RAG-oriented parsing option. Databricks became newly relevant once ai_parse_document reached general availability and made a single-function parsing path viable inside the lakehouse.
So the decision was not "cloud A versus cloud B." It was whether a RAG ingestion layer should remain an external document-analysis workflow or move into the same platform where the corpus, chunks, governance, evaluation, and vector index already lived.
Azure Content Understanding
For the Azure side, we used Content Understanding's prebuilt-documentSearch analyzer with API version 2025-11-01. That was the relevant Azure candidate because it is designed for document ingestion in RAG workflows and returns page-delimited markdown, figure descriptions, chart data, and page anchors.
The call shape is an external REST workflow:
For figure descriptions, Content Understanding uses a completion model deployed in the customer's Azure OpenAI account. In the main CU configuration, the prebuilt-analyzer-completion-mini alias was mapped to gpt-4.1-mini; in a follow-up CU configuration, we used gpt-4.1.
The important architectural point is that CU is not just a parser call inside the data pipeline. It is an external analysis workflow: render the file when needed, submit the bytes, poll the operation, collect markdown and figure outputs, and manage the completion model that writes descriptions.
Two operational details mattered in our run. First, CU needed rendered PDFs because native Office files did not paginate or produce figure analysis in the GA path. Second, figure descriptions depended on an Azure OpenAI deployment we had to capacity-plan separately.
Databricks ai_parse_document
The Databricks side used ai_parse_document version 2.0. The timing mattered: ai_parse_document entered Public Preview on October 23, 2025 and reached general availability on April 16, 2026. Until GA, an Azure-first parsing architecture was the reasonable default; after April 16, 2026, parsing could run where the lakehouse, governance, vector index, and evaluation already lived.
The parser ran as a Spark SQL expression over binary files:
The response is a typed document tree: text, titles, section headers, tables, captions, figures, footnotes, page ids, and AI-generated descriptions. Tables arrive as structure rather than flattened text. Figures arrive as retrievable content rather than invisible images.
Architecturally, this was the simpler candidate: parse bytes in Databricks, transform typed elements into chunks, write to Delta, and index. No document-by-document REST loop, no separate completion-model deployment for figure descriptions, and no Office-to-PDF requirement for Databricks-native parsing.
The governance angle also mattered. In RAG, the sensitive object is not only a table row. It is a chunk: extracted content, source document, page number, generated description, lineage, and permissions. Keeping that object inside the same governed data plane makes the answer to "who can see this chunk, and which document produced it?" easier to prove.
3. Methodology: One Moving Part
Frozen Stack, Parser Swap
To make the comparison useful, we removed every excuse except the parser. The retriever, embeddings, generator, chunking contract, corpus, and golden set stayed fixed. Only the document parser changed:
- same 40-document corpus
- same 33-question golden set
- same vector retriever with k=8
- same embedding model
- same generation model
- same page-level chunking contract for matched vendor comparisons
Why 40 Documents and 33 Questions
The test set followed two design rules:
- Hard on purpose: the 40 documents came from a much larger corpus, picked by scoring each against our catalogued failure modes: lost extractions, tables producing confidently wrong numbers, and chart-only slides. The mix was mostly PowerPoint, like the corpus itself, and ran to well over a thousand rendered pages.
- Small on purpose: every configuration re-parsed everything and evaluated three times, so corpus size multiplied straight into cost and time. Forty documents kept every configuration measurable.
The 33 questions were the curated golden questions those documents could answer, each with a known source document, page, and expected figure, so the pass-rates were checkable. And because the corpus was deliberately hard, absolute scores ran low: read the deltas.
The Five Metrics
Two were study-specific, judged against labeled expectations:
- Correct document in top-k: did the answer's source document appear in the k=8 retrieved chunks? This led the study because it is the least confounded measure of parse quality: a parser's job is to produce chunks that embed well enough for the right document to surface, and this metric sees exactly that.
- Numeric figure grounded: when the answer stated a number, could it be traced to a retrieved chunk? Much of what separated these parsers was numeric data inside charts and tables; this measured whether that data survived the full journey from extraction to answer.
Three came from the RAGAS evaluation framework:
- Answer Relevancy: did the response address the question? Thin or garbled chunks make the model hedge or decline; this turned that behavior into a number.
- Context Precision: of what was retrieved, how much was relevant? A parser that pads chunks with boilerplate or mis-merged fragments dilutes every retrieval it touches; the dilution shows up here.
- Faithfulness: was every claim in the answer grounded in the retrieved chunks? This guarded against a parser that looked good by producing confident answers the retrieved text did not support.
Read together, they formed a chain: did the right document surface, were its numbers traceable, was the retrieved set clean, did the model answer, and did it stay grounded while doing so.
Every reported number was a median of three end-to-end runs. The agent retrieved and answered non-deterministically, so a single run was not trustworthy. The LLM-judged metrics were also judged three times per answer, with the median taken.
We also kept a no-AI-parser baseline. It used local open-source extraction libraries and no figure descriptions. That gave us a floor: before choosing between vendors, we wanted to see whether an AI parser improved the retrieval substrate at all.
4. Results: What the Frozen Stack Measured
Did AI Parsing Beat the Baseline?
Yes, and by more than polish. ai_parse_document moved the numbers compared with local extraction.
What to notice: the largest gains are in correct source retrieval and numeric grounding.
The improvement was not just nicer extracted text. The correct source document surfaced about 55% more often. Numeric values were about 20% easier to trace to retrieved chunks. Answers became about 3% more relevant and about 6% more faithful. Averaged across the five metrics, AI parsing scored about 18% better than the no-parser baseline.
Which Parser Won the Matched Comparison?
This was the vendor decision. For the direct comparison, we used rendered PDFs on both sides because CU required PDFs for figure-aware analysis. The comparison became fair because both parsers received the same bytes.
What to notice: this is the cleanest vendor comparison because both parsers received the same PDF input and the same chunking strategy. The bars show how much more retrievable and groundable the evidence was with ai_parse_document.
This was the decision point. Averaged across the four metrics, ai_parse_document scored about 20% better than Content Understanding. ai_parse_document surfaced the correct document about 40% more often, grounded numeric values about 15% more often, retrieved about 13% cleaner context, and produced answers that were about 8% more faithful.
Did a Stronger CU Model Close the Gap?
No. We upgraded CU's figure-description model from gpt-4.1-mini to gpt-4.1, and the gap did not close.
What to notice: upgrading CU's description model helped parts of the run, but the main gap remained.
The CU upgrade improved parts of the pipeline, but it did not change the core decision. ai_parse_document still produced about 7% cleaner retrieved context and about 9% more faithful answers in this comparison.
Was Databricks Cheaper to Run?
Yes. The operational comparison pointed in the same direction.
What to notice: ai_parse_document's parser cost stayed below both CU configurations. CU became more expensive as figure descriptions moved through a separately billed completion model.
Cost mattered because the two providers made different things billable, and Databricks came out lower in the billed comparison. Per full-corpus build, ai_parse_document came to roughly two-thirds of what CU cost with the lighter description model, and roughly a third of what it cost with the stronger one. Averaged across both CU configurations, ai_parse_document ran at about half the cost.
The billing structures explain the gap. ai_parse_document's bill had one dial: pages, priced by complexity.
CU's bill had three dials:
- Page meter: the standard per-page parsing charge.
- Contextualization: Microsoft's per-page prep charge for its generative models.
- Description-model tokens: the open-ended dial, because figure descriptions billed through a separate Azure OpenAI deployment.
The cheap minimal meter never applied here: figure analysis required rendered PDFs, which billed on the standard meter.
On this corpus, the third dial dominated. Swapping the description model from gpt-4.1-mini to gpt-4.1 made the tokens five times more expensive and roughly doubled the total build, because the flat page meter diluted the ratio. With ai_parse_document, figure descriptions were part of the parser call, so there was no third dial to plan for.
The execution model followed the same pattern. ai_parse_document parsed inside the Spark job and fanned out across executors, so its throughput was the job's throughput. CU's throughput belonged to whichever Azure OpenAI deployment wrote the descriptions: scalable, but planned and paid for outside the parser call.
Normalised to the unit both vendors price in, ai_parse_document was the cheapest of the three by a wide margin, and on the CU side most of the difference was description tokens rather than the page meter.
5. Takeaways: What We Would Tell Another Team
The first lesson is diagnostic: if the right document is already in top-k and the answer is still wrong, stop treating the problem as retrieval-first. The plateau was the tell: retrieval tuning had nothing left to fix. Inspect the parsed chunks. The assistant could already find the report; the parser had not preserved what it needed to read.
The fastest test is to track two signals together:
The second lesson is methodological: a parser comparison is only as good as its matching. Feed both parsers the same bytes and build chunks the same way. For us, that meant rendering PDFs on both sides, since CU required them for figure-aware analysis. When a metric moved, the parser was the only thing that could have moved it.
The third lesson is the decision itself. Six things decided it:
- It won the matched pair. With identical PDF input and identical chunking, ai_parse_document surfaced the correct document about 40% more often and grounded numeric values about 15% more often than CU.
- The gap survived CU's model upgrade. Against CU with gpt-4.1 descriptions, ai_parse_document still retrieved about 7% cleaner context and produced about 9% more faithful answers.
- It was cheaper in the billed comparison. Per full-corpus build, ai_parse_document cost roughly two-thirds of CU with gpt-4.1-mini descriptions and roughly a third of CU with gpt-4.1.
- It fit the pipeline's execution model. ai_parse_document ran as a Spark SQL expression inside the Databricks job; CU remained an external analyze-poll-collect workflow with separate completion-model capacity planning.
- It kept the retrieval object in the governed data plane. Bytes to typed elements to Delta to index stayed on one lineage and permissions path.
- It gave us cleaner control over document structure. The typed element tree made table, chart, and page boundaries explicit instead of forcing us to recover those relationships downstream.
Before tuning retrieval again, run the two-signal check on your own pipeline: if retrieval is healthy and the answers still miss, the parser may be the ceiling.
Appendix: The Configurations We Tested
The 40 documents were a failure-weighted selection from the larger corpus, and the 33 questions were the curated golden questions answerable from them.
Beyond the parser itself, we varied CU's completion model. The matched-pair ai_parse_document run on rendered PDFs appears in the body's charts. Figure descriptions were indexed as separate chunks in every parser configuration. That produced four appendix configurations:
- Baseline (no AI parser): local open-source extraction libraries, no figure descriptions
- ai_parse_document, original files: ai_parse_document reading the native Office files
- CU, PDF, gpt-4.1-mini: Content Understanding on rendered PDFs with the mini description model
- CU, PDF, gpt-4.1: Content Understanding with the upgraded description model
