A Production RAG Pipeline for PDFs: Relational Parsing, TOC Retrieval, Typed Answers

The shape that survives the upgrade is the same four bricks Article 1 (minimal RAG) introduced. What changes is the contract per brick: what each one consumes, what it produces, and how the next one consumes it. The diagram below is the full contract: every brick, the outputs it produces, and which downstream brick consumes each one (including the parsing_summary side channel into both question parsing and generation). The per-brick subsections then zoom into each box.

Each of the four bricks is upgraded in its own articles, worth reading for the full contract:

Parsing runs once and turns the PDF into the small set of tables every later brick reuses.

In: pdf_path, the PDF on disk. Out: line_df, page_df, toc_df, parsing_summary.

What comes out, one row per unit:

• line_df: one row per visible line (page_num, line_num, text, bounding box). The citation unit.
• page_df: one row per page (page_num, text). The coarse scan surface.
• toc_df: one row per section (title, level, start_page). The document’s own map.
• parsing_summary: document-level metadata (doc type, language, page count, layout). The side channel into the LLM bricks.

The bootstrap chunk at the top of the article already produced the three DataFrames on the Attention paper. A look at each:

The page_num and line_num on each row are what turn an answer into a citation. Drawn back onto the page they came from, the rows are literal: every recognized line is one box, and its line_num sits in the gutter.

Aggregating the lines page by page gives page_df, the natural page unit that carries whole-page text and page-level context:

toc_df carries the native outline; on the Attention paper it has three levels and twenty-two entries, one row per section with its title, level, and start page:

Three tables, same shape contract, same numeric primary keys. Downstream bricks read what they need without re-parsing the PDF; Retrieval (Section 2.3) scans keyword hits and reads toc_df to anchor on the right section, then sizes the context around it (the whole section, or a line window) at the granularity the question implies. page_df is the page-level scan unit, toc_df the map, line_df the atomic lines the anchor and window are cut from. parse_pdf actually returns more in the same dict (image regions, internal references, named objects) plus a parsing_summary carrying document-level metadata (doc type, language, page count, layout, typical fields, a short summary); this section focuses on the three tables retrieval reads, and parsing_summary returns in the second part as the side channel that travels into the LLM bricks.

2.2 Question parsing: from noise to brief

Question parsing turns the raw user string into a typed brief the next two bricks can act on.

In: the raw question, the expert’s concept_keywords_df, and parsing_summary for document context. Out: a ParsedQuestion carrying intent, keywords, a RetrievalQuery brief, and a GenerationBrief.

What comes out:

• keywords: typos fixed and content terms pulled in one LLM call, then expanded with the expert’s vocabulary.
• intent: the answer shape the question implies (factual, listing, comparison).
• RetrievalQuery: the brief retrieval consumes (main_query, rewrites, anchor_keywords, section_hint, layout_hint).
• GenerationBrief: only the fields generation can act on (the question, the answer shape, disambiguation).

Article 1 (minimal RAG) called get_keywords_from_question on the clean question and got a corrected_question plus a short list of keywords back, in one LLM call. That single call does two jobs at once: it fixes the surface typos and pulls the content keywords. The corrected keywords come straight out of the parse, with no separate spell-checking pass bolted on afterward. If a keyword genuinely does not exist in the document, the recovery is retrieval’s feedback loop (Article 13, the workflow pipeline), which re-searches with the terms the document actually uses, not a blind snap onto the nearest corpus token.

What this article adds on top of the parse is the expert vocabulary layer. The corrected keywords are expanded with the domain terms a practitioner would also search for, pulled from the expert’s concept_keywords_df. Asked about positional encoding, the expansion adds the two concrete methods, sinusoidal and learned, so retrieval matches them even though the question never named them.

Each step is explicit, so retrieval downstream can use all of it and an audit log can be reconstructed.

The noisy question is the same one a frustrated user would type:

The one call already corrected the typos: optoins and posiitional came back as clean, content keywords, with no second spell-checking pass bolted on. Now expand them with the expert’s vocabulary, the concept_keywords_df table that maps each topic to the terms a practitioner would also search for:

{
  "original_question": "What are the optoins for posiitional encoding?",
  "keywords": ["positional encoding"],
  "expanded_keywords": ["positional encoding", "sinusoidal", "learned"]
}

Two things happened in one pass. The keywords came back corrected: posiitional and optoins were typos, and the single parse_question call fixed them while pulling the content noun phrase, dropping framing words like options. If a keyword still did not exist in the document, retrieval’s feedback loop (Article 13, the workflow pipeline) would recover it, not a blind snap onto the nearest corpus token.

The expanded keywords come from concept_keywords_df. Asked about positional encoding, the expansion adds the two concrete methods the paper uses, sinusoidal and learned, so retrieval matches them even though the question never named them. The table is small on purpose: each entry narrows on the topic, not a generic word like position that would pollute the search. Article 6 (question parsing) develops how it is built and maintained.

For the question-parsing code in detail, see the three articles that develop the brick:

• Article 6A (the thesis): parse the question before you search, the missing step in most RAG pipelines.
• Article 6B (extraction): the five fields the parser pulls from any question (keywords, scope, shape, decomposition, clarification).
• Article 6C (dispatch): what the parsed question decides downstream (chunk strategy, model tier, fragments, audit trail).

2.3 Retrieval: structured tables

Retrieval narrows the document down to the lines generation will read. It filters on the structured tables, it does not search a vector index.

In: the RetrievalQuery brief (from question parsing), plus line_df and toc_df (from document parsing). Out: a RetrievalResult: the kept pages and filtered_line_df, the section or line window generation reads.

How it works, in two phases:

• Anchor: keyword hits counted per TOC section, then the LLM TOC router reads the outline and picks the section that answers the question.
• Context: sized around the anchor at the granularity the question implies (the whole section for a listing, a line window for a pinpoint fact).
• filtered_line_df: just the lines generation will read, each carrying its page_num and line_num for citations.

Retrieval works in two phases (Article 7A, retrieval as filtering). First it finds the anchor: keyword hits, counted per TOC section, tell the router which sections carry the question’s terms; reason_on_toc reads the TOC plus those counts and picks the section. Then it sizes the context around that anchor, following the granularity the question implies (Section 2.2): the whole section for a listing or section question, or a line window around the match for a pinpoint fact. The section is the natural context unit; page_df is the coarse scan surface, line_df the atomic lines the window is cut from.

Here, to stay incremental on Article 1 (minimal RAG), this article runs the two detectors and merges their pages; the production hybrid routes them into sections instead (Article 7 (retrieval), the section-first arbiter above). The keyword method runs first (cheap, no LLM, deterministic). The LLM TOC router runs next on the same toc_df. The union of their pages goes to generation.

Article 7B (anchor detection) develops the LLM TOC router in detail (the prompt, the Pydantic output, why it beats substring matching on real documents). Article 7C (the LLM arbiter) completes the picture, ranking every candidate page from every detector in a single call. We use the standalone reason_on_toc here because it carries its weight on its own: the upgrade from substring is the single most impactful change a team running Article 1 (minimal RAG)’s pipeline can make to retrieval.

Article 7 (retrieval) introduces the unified retrieve_context(question, line_df, *, method, top_k, ...) → RetrievalResult dispatcher that routes to keyword / embedding / TOC / hybrid behind a single signature, and returns a typed RetrievalResult instead of two tuples. We use the raw retrieve_pages and reason_on_toc here to keep this article incremental on top of Article 1 (minimal RAG); production code calls retrieve_context or the shared dispatch_page_retrieval helper.

A useful check before trusting that table. The first matching line column scans line by line, one line at a time. That works for single-token keywords. A multi-word keyword like positional encoding can be broken across a line break in the PDF, with positional at the end of one line and encoding at the start of the next. Each line on its own contains neither phrase. The line-by-line scan finds nothing, even when the keyword is right there on the page.

Count the misses across the document:

The line is a PDF rendering artifact. The text the parser sees as one line is whatever fits between two visual line-break decisions made by the PDF generator. There is no semantic reason the unit of detection should map to that arbitrary boundary. The fix is to detect on a passage instead, where a passage is a small window of adjacent lines joined with a space. Any keyword that exists in the passage will be found, no matter where the line breaks fall.

The page-level match_count was already correct, because retrieve_pages joins all lines on a page before scanning. The fix targets the line-grained helpers that the snippet column, the highlighting, and any later chunking step need. From here on, every helper that scans below the page level uses a passage window, not a single line.

The LLM TOC router fills the other gap exposed above. Same toc_df from parsing, but the LLM reads it whole and reasons about which section answers the question, returning the picked section ids plus a one-sentence rationale. The function itself is short (format each TOC row, drop it in a prompt with the question, parse a typed SectionSelection back); Article 7B (anchor detection) shows it in full.

Run on the noisy question against the paper’s 22 TOC entries:

One LLM call, the whole TOC inside the prompt, a typed list of section_ids back with a sentence of reasoning. The substring matcher this article used to ship would have caught 3.5 Positional Encoding on this specific question because encoding appears in the title. It would have missed every question phrased differently from the author. “What happens if we exit early?” against a contract whose section is titled “Termination”: substring zero, LLM one. The cost is a small LLM call (a few thousand tokens for a typical TOC, a few hundred milliseconds), and it is cached forever on identical inputs.

This view is what an expert reads. Not a flat page list with cosine scores, but the document’s own outline, with the parsed-question keywords marked where they land. Article 7C (the LLM arbiter) consumes exactly this shape as a structured brief, one row per candidate, and ranks them with per-candidate roles + reasons. That arbiter is an upgrade on top of the LLM TOC router walked above, kept out of scope here to stay incremental on Article 1 (minimal RAG).

For the retrieval code in detail, see the three articles that develop the brick:

Generation fills a typed schema from the retrieved lines. It is controlled execution against a contract, not free-form prose.

In: the GenerationBrief (question and answer shape, from question parsing), filtered_line_df (from retrieval), and the answer schema. Out: a typed AnswerWithEvidence, or a ListAnswer when the question asks for a list.

What comes out:

• answer: shaped to the question, one string for a single fact, one entry per item for a listing.
• evidence_spans: a line range plus a verbatim quote per item, checkable against filtered_line_df.
• quality signals: confidence, caveats, and a context_structured flag the LLM sets when the retrieved text no longer reads in order.

Article 1 (minimal RAG)’s AnswerWithEvidence returns one answer: str, one contiguous evidence span, a few quotes. Good for a question with a single answer (“What dataset was used?”). The running question asks for options, plural. The baseline schema can list them inside the string, but the caller has no clean way to iterate over the items or to attribute one citation per option.

Article 8 makes the schema match the shape of the expected answer. When the question parser flagged expected_answer_shape = "listing" (Article 12 (listing) develops these in depth), the generation brick returns a ListAnswer with one entry per item, each carrying its own evidence span and its own verbatim quote.

The shape is what matters: one AnswerItem per option (its text, a start/end line-range span, and a verbatim quote), wrapped in a ListAnswer that also carries the answer-quality flags (answer_found, complete_answer_found, context_structured, confidence, caveats). Article 8 (generation) develops the schema in full.

Filled in on the noisy positional-encoding question, the schema returns two items and the full set of quality indicators:

{
  "items": [
    {
      "text": "Sinusoidal positional encodings (sine and cosine functions of different frequencies).",
      "start_page_num": 6, "start_line_num": 33, "end_page_num": 6, "end_line_num": 37,
      "quote": "we use sine and cosine functions of different frequencies"
    },
    {
      "text": "Learned positional embeddings.",
      "start_page_num": 6, "start_line_num": 39, "end_page_num": 6, "end_line_num": 41,
      "quote": "we also experimented with using learned positional embeddings"
    }
  ],
  "answer_found": true,
  "complete_answer_found": true,
  "context_completeness": 1.0,
  "context_structured": true,
  "confidence": 0.98,
  "caveats": []
}

The four indicators carry the answer’s trust profile. complete_answer_found says whether the answer covers every option the document mentions or only a subset. context_completeness says how well the retrieved lines covered the question, separate from whether the answer itself is right. context_structured flips to false when the LLM cannot follow the reading order, the canonical OCR-failure signal. A downstream router reads them: high confidence + complete + structured = ship the answer; low completeness or unstructured context = retry retrieval, or fall back to a deeper parse (the path Article 10 develops).

context_structured is the indicator the article does not get to see fire on a clean paper. To prove the LLM uses it, the same lines fed in random order should flip it to false:

{
  "clean context": {
    "answer_found": true,
    "complete_answer_found": true,
    "context_completeness": 1.0,
    "context_structured": true,
    "confidence": 1.0,
    "n_items": 2,
    "caveats": []
  },
  "shuffled context (same lines, random order)": {
    "answer_found": true,
    "complete_answer_found": true,
    "context_completeness": 1.0,
    "context_structured": true,
    "confidence": 0.95,
    "n_items": 2,
    "caveats": []
  }
}

In production, scrambled context comes from a different source than df.sample. PDFs with two-column layouts that the parser read column by column instead of row by row, scanned documents OCR’d by a model that lost the layout, exports from Word that broke long tables across hidden anchors. The pipeline cannot self-correct on those, but the answer schema tells the caller something went wrong, and a separate code path takes over.

Compare with the baseline RAG output. Same question, same paper, but the caller now gets a structured object: number of items known up front, each item independently citable, four quality indicators that route the answer to the right next step. A UI renders the items as a clickable list. A SQL pipeline writes one row per item. An annotated PDF highlights each quote on its source page. None of those consumers had to parse prose, and none of them ships a confidently-wrong answer because the schema makes the failure modes visible.

Four bricks, four upgrades. Side by side, the same pipeline before and after, one brick per row:

The four upgrades are independent. A team running Article 1 (minimal RAG)’s pipeline can adopt them one at a time: a TOC at parsing, a corpus-vocab spell-check at question parsing, an LLM TOC router + keyword fallback at retrieval, a list-shaped schema at generation. Each one is a small drop-in, none of them rewrites the surrounding code.

The four bricks now speak in typed contracts:

• Parsing emits a relational set, not a flat dump.
• Question parsing emits a brief, not a bag of words.
• Retrieval emits a merged page set backed by the document’s own structure, not a top-k guess.
• Generation emits a typed answer with a span per item, not a paragraph the caller has to read again.

Each upgrade earned its place against a concrete failure of the baseline: the typo that missed the page, the TOC the keyword search ignored, the list flattened into prose. What the bricks do not have yet is a single entry point and a feedback path between them. The second part wires them into one call and runs it on real documents, including one whose TOC is broken, in Composing the four RAG bricks into one pipeline, tested on real documents (link to come).

It helps to see where this pass sits on a longer climb. The same four bricks stay fixed; what changes from one rung to the next is where the control lives. This article is rung two: pdf_qa, a single richer pass that already emits the feedback fields (retrieval confidence, missing keywords) but does nothing with them yet. Article 13 turns that pass into pdf_qa_flow, the Volume 1 composite that dispatches on question patterns and re-runs the pass in a bounded loop, acting on exactly those fields. Volume 2’s pdf_chat puts a multi-intent entry in front. Only the last rung hands the control loop to the LLM itself.