# Extraction Pipeline — Canonical `tuned-financial` Configuration

This document describes the default docling configuration Quber uses for PDF
extraction, the rationale, and the known gaps that this configuration does
**not** address.

The configuration was selected by [QUE-217](https://mandeng.atlassian.net/browse/QUE-217)
after a 10-configuration evaluation against the TMUS Q2 FY25 Investor Factbook
(SEC EX-99.2), and adopted as the Quber default by [QUE-218](https://mandeng.atlassian.net/browse/QUE-218).

## Canonical Configuration

The `tuned-financial` preset is the default for both the parser layer and the
`quber parse` / `quber analyze` CLI commands. The configuration is a *type*,
not a runtime flag (QUE-229): `TunedFinancialParser` is the canonical default
and `LegacyDoclingParser` reproduces the pre-QUE-218 behaviour. Both subclass
the abstract `DoclingParser`. Select one on the CLI with
`--preset [tuned-financial|legacy]` (default `tuned-financial`), or in code via
`parser_for_preset("legacy")`.

### Pipeline options (applied inside `TunedFinancialParser`)

| Knob | Value | docling API |
| --- | --- | --- |
| Table structure mode | `ACCURATE` | `pipeline_options.table_structure_options.mode = TableFormerMode.ACCURATE` |
| Cell matching | `True` | `pipeline_options.table_structure_options.do_cell_matching = True` |
| Table structure | `True` | `pipeline_options.do_table_structure = True` |
| OCR | `True` | `pipeline_options.do_ocr = True` — matches the canonical CLI default. Docling only triggers OCR for pages without a text layer or for graphical tiles where text is embedded in images (e.g. the page-3 highlight tiles on the TMUS factbook). It is **not** force-OCR; clean text layers are unaffected. |
| Picture classification | `True` | `pipeline_options.do_picture_classification = True` |
| Generate picture images | `True` | `pipeline_options.generate_picture_images = True` |
| PDF backend | `DoclingParse` (explicit) | `PdfFormatOption(backend=DoclingParseDocumentBackend, ...)` — what the canonical CLI's `--pdf-backend dlparse_v4` normalizes to in docling 2.93+ (the `dlparse_v4` enum value is a deprecation alias for `docling_parse`). |
| Accelerator device | `AUTO` | `accelerator_device: AcceleratorDevice = AcceleratorDevice.AUTO`, resolved at run time by docling's `decide_device`. The CLI's `--device` default is `auto`. |
| Accelerator threads | `os.cpu_count()` | `pipeline_options.accelerator_options.num_threads = N` |
| Page batch size | `32` | `docling.datamodel.settings.settings.perf.page_batch_size = 32` (process-wide singleton) |

The page batch size is a global module setting on docling's `settings`
singleton, **not** a field on `PdfPipelineOptions`. `DoclingParser.parse`
assigns it before invoking the converter. At 32 (vs. docling's default of 4),
each batch amortises model load/unload over 8× more pages — this is the
runtime gain observed in the QUE-217 evaluation.

### Export-time options (applied at the CLI layer)

`quber parse` mirrors the canonical CLI's export defaults:

- `--image-mode referenced` — matches `--image-export-mode referenced`
- `--html` — opt-in for `html_split_page` export, matching `--to html_split_page`
- `--format both` (default) — markdown + JSON, matching `--to md --to json`

### Equivalent docling CLI invocation

The configuration is intended to be bit-equivalent to:

```bash
uv run docling "$PDF" \
  --device cuda \
  --pipeline standard --table-mode accurate \
  --pdf-backend dlparse_v4 \
  --enrich-picture-classes \
  --page-batch-size 32 --num-threads $THREADS \
  --image-export-mode referenced \
  --to md --to json --to html_split_page \
  --output "$OUT"
```

## Known Gaps (NOT fixed by this configuration)

The `tuned-financial` configuration was selected because it has **zero
transcription drift on captured content**. It does not, however, fix the
following docling-level silent gaps surfaced in QUE-217 against
`documents/TMUS_Q225_992.pdf`. All five gaps are on **selectable text-layer
content** — none are in image regions — and all five reproduced across every
configuration evaluated, meaning they are upstream docling behaviours rather
than artefacts of our pipeline choices.

| Page | Defect |
| --- | --- |
| 3 | Footnotes (2) and (3) bodies dropped |
| 14 | Footnote (1) and the Metronet/UScellular scope line dropped |
| 21 | Postpaid ARPA `$148.06` mislocated (hoisted into header row) |
| 27 | Definitions items 2 (Customer) and 3 (Churn) dropped |
| 29 | Cautionary Statement + About T-Mobile bodies dropped (highest severity) |

Annotated PNGs of each defect are attached to QUE-217.

These gaps are deliberately **out of scope** for QUE-218. They are tracked in
[QUE-216](https://mandeng.atlassian.net/browse/QUE-216), which will surface
them as structured `quality_flags` via a raster flag-only post-processing
pass.

## Why VLM and enrichment pipelines were ruled out

The full ranking is in QUE-217. In short:

- VLM pipelines (`vlm-granite-docling`, `vlm-smoldocling`) — disqualified
  outright (`AT&T → ATAT`, hallucinated Q3/Q4 2025 columns, generation collapse).
- `--enrich-chart-extraction` — silently drops charts and mislabels stacked
  bars (120× overstatement risk observed).
- `--enrich-picture-description` — invented prose captions ("2,451%", etc.).
- `--force-ocr` — introduced ~40 distinct drifts including `$33.2 → 833.3` on
  documents with a clean text layer.
- `--table-mode fast` — structural failures on financial tables.

None of these enrichments are enabled in `tuned-financial`. The picture
**classification** enrichment is enabled (different feature, no observed
hallucinations).

## Logfire fingerprint

Each `DoclingParser.parse` call emits a `docling_parser.parse` Logfire span
with the resolved pipeline-option fingerprint as attributes (preset, table
mode, backend, accelerator device, num_threads, page_batch_size, etc.). This
provides post-hoc confirmation of which configuration produced a given run.

## Extraction-pipeline orchestration

The table engine's orchestration is documented in
[`docs/SET_OF_MARK_EXTRACTION_GRAPH.pdf`](SET_OF_MARK_EXTRACTION_GRAPH.pdf):
the graph topology, the agents, the value guards, and the concurrency model.
That document is rendered from the code and is the reference for this layer;
this file does not restate it.