j%SrSSKJr SSKrSSKrSSKJr SSKJrJ r J r J r J r J r Jr SSKJrJrJr SSKJr SSKJr \"S S \ 55r"S S \5r"S S\5r"SS\5r"SS\5r"SS\5r\"SSSS9\"SSSSS9\"SSS S!S9\"S"S#S$S%S9\"S&S'S(S)S*S+9\"S,S-S.S)S/S+9\"S0S1S2S)S39\"S4S5S6S)S39\"S7S8S9S)S39\"S:S;SS?S)S39\"S@SASBS)SCS+9\"SDSESFS)S39\"SGSHSIS)S394rSJ\SK'\"1SLk5r \"SM\55r!"SNSO\5r""SPSQ\5r#SXSRjr$"SSST\5r%SYSUjr&SZSVjr'S[SWjr(g)\ag TableExtractor Protocol and ExtractedTable Pydantic model. `ExtractedTable` carries the LLM-correction outputs (title, caption, markdown, footnotes, footnote refs and marks, units, header rows) plus the Camelot-side provenance (page, bbox, flavor). `subtitle` is always left empty and exists only so older artifacts still load. The numeric content lives inside `markdown`. Where its values come from depends on the path, and the system prompts in `quber.agents.llm_client` set the rule for each. The deprecated camelot-llm correction may not change any Camelot value. The set-of-mark correction takes values from Camelot's grid and the table's text layer, and may recover rows Camelot missed from that text layer. A table the figure stage reads off a page image (`kind` image_table) involves no Camelot: its grid, fingerprint and `som_region` come from the page scan. ) annotationsN)Path)ListLiteralOptionalProtocolSequenceTupleruntime_checkable) BaseModelFieldfield_validator)ClassifierResult) FootnoteDefc(\rSrSrSSjrSSjrSrg)TableExtractor c # g7fNselfsources 8/home/mande/repo/quber/src/quber/core/extractors/base.pyextract_tablesTableExtractor.extract_tables"s#scgrrrs rextract_tables_sync"TableExtractor.extract_tables_sync$srN)rrreturnzList[ExtractedTable])__name__ __module__ __qualname____firstlineno__rr__static_attributes__rr rrr sMLr rc\rSrSr%\"SSS9rS\S'\"SSS9rS\S'\"SS S9rS\S '\"SS S9r S\S '\"SS S9r S\S'\"SS9r S\S'\"\ SS9r S\S'\"\ SS9rS\S'\"\ SS9rS\S'\"SSS9rS\S'\"SSS9rS \S!'\"S"S9rS#\S$'\"SS%S9rS&\S''\"SS(S9rS&\S)'\"SS*S9rS&\S+'\"\ S,S9rS-\S.'\"\ S/S9rS-\S0'\"\ S1S9rS2\S3'\"\ S4S9rS\S5'\"S6S7S9rS8\S9'\"S:S;S9rS<\S='\"SS>S9rS?\S@'\"SASASBSCSD9rSE\SF'\"SSGS9rSH\SI'\"SJSKS9rSL\SM'\ "SSNSO9\!SUSPj55r"\"SSQS9r#SR\SS'STr$g)VExtractedTable'aDeterministic address of this table: '-p-t', where is the source file stem, the 1-indexed page, and the locator's reading-order ordinal on that page. A sub-table produced by splitting a fused region appends '-s'. The same document yields the same IDs on every run, so external references (a review comment, a validation report, a UI link) stay valid across runs. Empty for engines that do not set it.default descriptionstrtable_idzFirst 8 hex characters of sha256 over Camelot's raw value grid. The same document yields the same fingerprint on every run; a changed value at the same table_id shows as a different fingerprint. Empty when no grid backed the table.content_fingerprintzIThe table's printed name, copied off the page; empty when none is printedtitlezLThe sentence that introduces the table, copied off the page; empty when nonecaptionz5Retained for older artifacts; new runs leave it emptysubtitlez5Corrected markdown table; numeric values from Camelotr-markdownzFootnotes printed below the table, each a pair of the footnote's own marker (empty for an unmarked general note) and its text. Read off the table image by the correction agent; the image's formatting decides what is a marker.default_factoryr-zList[FootnoteDef] footnotesuFootnote reference markers carried on the table's headers or cells, read off the table image — a superscript or parenthetical number, letter, or symbol that points to a footnote: e.g. '1', '(1)', a letter 'a', an asterisk '*', a dagger '†' or double-dagger '‡', or a section sign '§'. Written as it appears. Empty if the table carries none. Survives superscripts the text layer drops; the demand-driven footnote lookup keys off this. List[str] footnote_refsuWhere each footnote reference marker sits: one entry per carrying cell, addressed by row/col in the corrected markdown (header row 0). The agent QUOTES the carrying cell and a lookup resolves the quote to its cell — it never counts positions. An entry with row/col None is a marker whose quote matched no cell (or that was catalogued without a quote) — kept and flagged, never dropped.zList['LocatedMarker']footnote_marksz6Scale/currency unit attribution caption; empty if noneunitsNzLeading column-header rows of the corrected markdown, counted off the table image by the correction review. A stacked header counts every stacked row; 0 means the fragment starts at data. None on artifacts extracted before this capture existed. Optional[int] header_rowsz1-indexed page numberintpagez4Camelot's geometric box on the page (x1, y1, x2, y2)+Optional[Tuple[float, float, float, float]]bboxzSet-of-Mark vision boundary box that located this table, normalized 0..1 with the page top-left as origin (x1, y1, x2, y2). Full-precision floats, never floored or snapped to a grid cell. None for engines that do not run the Set-of-Mark locator. som_regiona The table's actual content extent, normalized 0..1 top-left origin (x1, y1, x2, y2). Same box as som_region but with the bottom revised down to the last tabular row, located by matching the corrected markdown's last row back to the page text layer. som_region and bbox can overrun past the grid (e.g. enclosing footnote lines printed below it); this is the authoritative end of the table. Equal to som_region when no overrun is found or the last row could not be located. None for engines without the Set-of-Mark locator.content_regionubThe Camelot cell grid with per-cell geometry, as extracted before LLM structure correction. Rows and columns align with Camelot's raw grid, not the (possibly restructured) `markdown` — values are preserved verbatim through correction, so the geometry can be remapped onto the corrected cells by value later. Empty when no Camelot grid backed the table.zList[List['GroundedCell']] cell_griduThe corrected markdown's grid with per-cell geometry — one GroundedCell per markdown cell, same row/col addressing (header row 0), boxes normalized 0..1 top-left. This is the complete cell-level view, fully resolved at the grounding stage: combined cells carry the box their merge's reported addresses resolved to, and every other cell its Camelot cell box via order-preserving alignment to the raw grid — whole-cell matches first, then fragments of glued Camelot cells by ordered span tiling within the aligned rows. box is None only where a cell's text has no home in the raw grid; those gaps are warned in the run log at extraction. Equal to cell_grid when no correction was applied.corrected_grida(Corrected cells that combine two or more source cells (a rejoined split symbol, or a flattened multi-row/spanning header), each addressed by its row/col in the corrected markdown and resolved to one Camelot cell box per reported source address plus their union. Empty when no cells were combined.zList['MergedCellBox'] merged_cellsu.Text printed in the table's header area that appears in no output cell — usually surrounding page text the correction rightly excluded, but recorded so an omission is never silent. Each entry becomes a table-level review flag (status 'header_text_dropped'). Empty when everything printed was carried. dropped_textunknownz2Camelot extraction flavor that produced this tablez'Literal['lattice', 'stream', 'unknown']flavor text_tableaWhat this region really is, set by reconciliation against docling. text_table: a real text-layer table (the default). chart: SoM mistook a docling-classified picture (bar/line chart, etc.) for a table. image_table: a real table rendered as an image (no text layer under a docling-detected table, Camelot empty). Its content is whatever last read the image: the page-scan workflow's reading where that has run, and docling's own OCR of it where it has not.z-Literal['text_table', 'chart', 'image_table']kindzSource PDF path Optional[str]rggY@zSCamelot parsing_report.accuracy (0-100); how cleanly text snapped to detected cells)r,geler-floatcamelot_accuracyzVWhat the TableClassifier decided about this candidate; None if classifier was bypassedzOptional[ClassifierResult]classifier_decisionFzJTrue if the LLM modified Camelot's raw markdown during the correction passbool llm_correctedbefore)modec[U[5(a-UVs/sHn[U[5(aSUS.OUPM! sn$U$s snf)uArtifacts extracted before footnotes carried their marker stored each footnote as a bare string. Load those as unmarked notes — the text is intact; only the marker pairing is absent — so old artifacts stay readable.r*)markertext) isinstancelistr.)clsvitems rcoerce_legacy_footnote_strings-ExtractedTable.coerce_legacy_footnote_stringssJ a  `ab`aX\JtS4I4Ir40tS`ab bcs&AzIRecord of the detector/correspondence/completeness flow; None if not usedzOptional[ExtractionRecord]extraction_recordr)r]objectr!rb)%r"r#r$r%r r/__annotations__r0r1r2r3r5r[r8r:r;r<r>r@rBrCrDrErFrGrHrJrLrrQrRrTr classmethodr_rar&rr rr(r('s : Hc  % ^  kE3 nGS"2ijHcj&]^Hc^#( X$I  % $ M9 -2 * -N) r/ghE3h!& "K"9:D#:8="X9D 5?D S?J;CH _ CN? -2 Q -I) 27 A 2N. +0 [+L'$ ^L)7<'[7F 3;@ < ;D 7 "$Camelot candidate_ids assembled into this table, in page orderr6r9source_candidate_idsFzTrue if a single Camelot chunk spans two or more detected tables (convergence). The chunk is kept whole and flagged, never split.rScombinedzGDetected ordinals a combined chunk spans; empty unless combined is Truez List[int]combined_ordinalszWcompleteness audit verdict; None if the audit did not run (e.g. detected_not_extracted)zOptional[bool]completeness_completezVIf truncated, which edge was cut off (per the completeness audit); empty when completecompleteness_gapzTrue if the text-layer fill step inserted one or more figures to round off an edge truncation. The inserted values are listed in filled_cells; every other cell remains Camelot-sourced.filled_from_text_layerzvFigures inserted by the fill step, with their grid position and text-layer origin; empty unless filled_from_text_layerzList[FilledCell] filled_cellsr)r"r#r$r%__doc__r rhrcrirjrkr[rlrmrnrorprqrrr&rr rrfrfs/"LQ = LF H ',^'m!&j!#>Ca>N:',T') OHd$)]$y-2m->"lc$) '$D&+M&L"r rfc\rSrSr%Sr\"SS9rS\S'\"SS9rS\S '\"S S S 9r S\S '\"SS9r S\S'\"SS9r S\S'\"SS9r S\S'\"SSS 9r S\S'Srg) FilledCelli1aOne figure inserted into a table by the text-layer fill step. Records that this value did NOT come from a Camelot chunk: it was read directly from the PDF text layer at (x, y) and placed into the grid to round off an edge truncation. Camelot remains the source for every other cell; this keeps the amendment auditable and never blurs a filled value with a Camelot-extracted one (the source field is always `text_layer`). z/The literal text-layer figure that was insertedr4r.valuezY0-based column index in the assembled row the value was placed in (0 is the label column)r?columnr*z3Label of the restored row the value was placed intor+ row_labelz,Which edge the truncated row was restored atzLiteral['top', 'bottom']edgez'Text-layer x of the figure (PDF points)rPxz'Text-layer y of the figure (PDF points)y text_layerzIAlways text_layer; distinguishes a filled cell from a Camelot-sourced onezLiteral['text_layer']rrN)r"r#r$r%rsr rvrcrwrxryrzr{rr&rr rruru1s#TUE3UoFC23hiIsi%*7e%fD "f!JKAuK!JKAuK$)_%F !r ruc\rSrSr%Sr\"SS9rS\S'\"SSS S 9rS \S '\"SSS S 9r S \S'\"SSS9r S\S'\"SSS9r S\S'Sr g) LocatedMarkeriJuOne footnote reference marker tied to the cell that carries it. `row`/`col` address the carrying cell in the corrected markdown (header row 0), resolved by looking up the cell text the agent QUOTED for the marker — the model never counts positions. Both None when the marker is known to be on the table but its carrying cell is not — the marker is kept and flagged for review rather than guessed onto a row. z?The reference marker as printed: '1', '(1)', 'a', '*', a daggerr4r.rXNrzN0-based row of the carrying cell in the corrected markdown; None when unplaced)r,rNr-r=rowzQ0-based column of the carrying cell in the corrected markdown; None when unplacedcolfootnoteuSThe correction agent's judgement of what the marker points at, made from the page image: 'footnote' — a note printed for this table; 'section' — a cross-reference to a named part of the document whose content lives elsewhere. Resolution routes on this: section references resolve to a heading pointer and are never hunted as footnotes.r+zLiteral['footnote', 'section']rLcellu'cell': the marker is carried by a grid cell at row/col. 'table': the marker is carried by the table's own title or subtitle text — a spanning band or title suffix — so it qualifies the whole table and reaches every record; row/col are None but the marker is placed, not flagged.zLiteral['cell', 'table']scoper) r"r#r$r%rsr rXrcrrrLrr&rr rr~r~Js$efFCf dC  gC ,1 A ,D ( ', N'E #r r~cT\rSrSr%SrS\S'S\S'S\S'SrS\S 'S rS\S 'S rg ) CellStatusitasOne entry of the cell-status reference: the machine code a corrected cell carries and the human framing a front end shows for it. `inspect` marks the codes a consumer surfaces to the user when the table is used. `cases` names the residue-catalog cases the code covers (the catalog is docs/CELL_STATUS_CASES.pdf), so a status always points back to a demonstrated example. r.codelabelr-FrSinspectr*casesrN) r"r#r$r%rsrcrrr&rr rrrts, I JGTE3Or r reconciledzMatches the page.z]The cell's text tied back to a printed source and carries that source's measured coordinates.)rrr-header_printed_unlocatedz?This header is printed on the page. We could not measure where.uHeader or band text is printed on the page, but its coordinates could not be measured. The text is correct; only the geometry is missing. Expected behavior — a pass, not a defect.1)rrr-rlabel_printed_unlocatedzBThis row label is printed on the page. We could not measure where.uA row label is printed on the page, usually wrapped across printed lines, but its coordinates could not be measured. Expected behavior — a pass, not a defect.z2 and 3single_characterzNA one-character cell, such as a bare "$". Not matched to a position by design.uOne-character text (a bare '$'), excluded from location matching by design: a bare symbol or digit would anchor inside any unrelated number. Working as designed — a pass, not a defect.7total_label_addedz@The page prints no label on the totals row, so we wrote "Total".uThe totals row is printed with no label; the extraction wrote 'Total'. Flagged for user review because the author could have intended something else — a review item, NOT a defect.T5)rrr-rrheader_label_addedz;The page prints no header over this column, so we named it.uThe table is printed with no header over this column; the extraction wrote a generic column name ('Item', 'Description'). Flagged for user review because the author could have intended something else — a review item, NOT a defect.6 unverifiedzUThis value is on the page, but our check of it did not complete. Worth a manual look.uThe condition could not be confirmed: the text has no printed source and is not an authorized conventional label, or the inspector could not tell from the image. An open question for user review — not yet a defect.)rrr-rdefectzGThe page shows something different from what we recorded. See the note.uThe inspector positively observed the page showing something OTHER than what the extraction recorded — a different word printed at an added label's position, or a structure the condition misdescribes. A confirmed defect; the evidence names what the page shows.footnote_unresolvedzQThis cell has a footnote marker, and we could not find the footnote it refers to.uA marker is printed on a table cell or a figure's label and no tier found the note it points at — not the pairs the correction agent read off the image, not the reading-order scan, not the demand-driven lookup. The element states a figure the document qualifies somewhere, and the qualification is missing: read alone it looks unqualified, which is indistinguishable from a clean reading. An element-level flag naming the marker, not an output cell.footnote_unreferencedz1A footnote on this page that no marker points to.unA note printed for a table or figure that no marker on it names. Currently judged within one element type, so a note a table claims can still read as unreferenced by a figure on the same page and the reverse — which is why it is catalogued but not surfaced. Becomes meaningful once every reader's claims are pooled and the leftovers reported once for the document.footnote_marker_unplacedzThe page prints a different figure here. Check which is right.aA chart value where the page fragment at the value's position states a different number than the reading. One of the two is wrong and the position pins exactly where to look, so the value never passes silently. A figure-value flag: it names the value and carries the fragment's box.value_unreconciledzFRead from the chart once, without a second check. Worth a manual look.a%A chart value present in one reading with no counterpart in the other. The note states the direction: a scan-only value the local read did not corroborate, or a locally read value the scan omitted. Either way the value stands unverified and is surfaced rather than passed. A figure-value flag.zTuple[CellStatus, ...]CELL_STATUS_REFERENCE>r^totalr-c#8# UHoRv M g7fr)r.0ss r r/sE/D!vv/Dsc\rSrSr%Sr\"SS9rS\S'\"SSS 9rS \S '\"SS S 9r S \S'\"SSS 9r S \S'\ "S5\ SSj55r Srg) GroundedCelli2uJOne Camelot cell paired with its geometry. `text` is the cell value (immutable through the pipeline). `box` is the cell's box normalized 0..1 with the page top-left as origin — the same frame as `som_region` and `MergedCellBox`, so every box the table carries reads in ONE frame — or None when no Camelot box backs the cell. zCell value from Camelot's gridr4r.rYNzOCell box (x1, y1, x2, y2) normalized 0..1, top-left origin; None if unavailabler+rAboxuWhy the cell does or does not carry a box; set on corrected-grid cells only. One of the codes in CELL_STATUS_REFERENCE — see that registry (and the set_of_mark README) for each code's meaning and whether a consumer should surface it for user inspection.rMrhzInspection evidence for a flagged cell: the status inspector's one-line account of what the table image shows at this position. None when the cell needed no inspection.notec@UbU[;a[SU<S35eU$)Nzunknown cell status z&; register it in CELL_STATUS_REFERENCE)_CELL_STATUS_CODES ValueError)r\r]s rstatus_is_a_registered_code(GroundedCell.status_is_a_registered_codeSs, =Q&883A58^_` `r r)r]rMr!rM)r"r#r$r%rsr rYrcrrhrrrdrr&rr rrr2s"BCD#C7<e8C 4" .FM  )D-Xr rc\rSrSr%SrS\S'S\S'SrS\S 'S rS\S 'SrS \S 'Sr S \S'S\S'S\S'Sr S\S'Sr g)CellFlagi[u,One item a run surfaces for review — the error-check record. Carries full document identity (source, page, table) alongside the item, so a flag can be traced to its table without reverse attribution from page numbers. `status` is a registered inspect code; `note` is the status inspector's one-line account of what the page image shows. Most flags point at an output cell; a table-level flag (status `header_text_dropped`, printed text that reached no output cell) has no cell to point at, so `row` and `col` are None and `text` holds the missing text. r.rr?r@NrMr/r*r1r=rrrYrhrr) r"r#r$r%rsrcr/r1rrrr&rr rrr[sN  K I"Hm"E3OCC I KD-r rc[S[55n/nUGHn[UR=(d /5HupE[U5HupgURU;dMUR [ UR=(d SURURUR=(d SUUURUR=(d SURS9 5 M M URHZnUR [ UR=(d SURURUR=(d SUSS95 M\ URHn U RbMU R S:wdM$UR [ UR=(d SURURUR=(d SU R"SS95 M GM U$)uEvery corrected cell whose status is registered for inspection, plus each table's dropped printed text and each footnote marker not tied to a cell, with document identity attached — the run's review queue, ready to emit.c3^# UH#oR(dMURv M% g7fr)rrrs rrcell_flags..wsQ.Cyyfaff.Cs--r*) rr@r/r1rrrYrhrr)rr@r/r1rYrhtabler) frozensetr enumeraterFrhappendrrr@r/r1rYrrHr;rrrX) tables inspect_codesouttrrcrfragmentmarks r cell_flagsrrss Q.CQQMC  0 0 6B7FA$S>;;-/JJ #$88>r!"%&ZZ"#''-R ! !!%#';;#4"!%  *8 H JJ88>rZZ''-R!0  '$$DxxDJJ'$9  xx~2VV!"ggm![[9  %9P Jr c\rSrSr%Sr\"SS9rS\S'\"SSS 9rS \S '\"SS S 9r S \S '\"\ SS9r S\S'\"\ SS9r S\S'\"\ SS9r S\S'\"SSS9rS\S'\"SSS9rS\S'S rg)! MergedCellBoxiuA corrected cell built by combining two or more source cells, resolved to per-source-cell boxes plus their union. The correction agent reads the Camelot grid inside a printed spreadsheet coordinate frame and reports each merge's source cells by ADDRESS (`source_cells`, e.g. ['B2', 'B3']); `source_boxes` holds one box per address, in the same order — the named cell's Camelot-measured box, or None where the address could not be resolved (bad label, blank cell, or text that does not appear in the result). `box` is the union of the located boxes, a coarse envelope over the whole combined cell. All boxes are normalized 0..1 with the page top-left as origin — the same frame as `som_region`. Every box is Camelot's own measurement; the model only ever repeats printed labels. z?The combined cell text, as it appears in the corrected markdownr4r.resultru0-based row of this cell in the corrected markdown table (header row is 0) — the exact address of the cell, since result text alone repeats (e.g. an identical total in the Basic and Diluted EPS rows))rNr-r?rz;0-based column of this cell in the corrected markdown tablerz)The source cell values that were combinedr6r9sourceszThe printed coordinate of each combined Camelot cell as the agent read it off the coordinate frame (e.g. ['B2', 'B3']); resolves into cell_grid by lookup source_cellsz\One box per source_cells entry (same order), normalized 0..1 top-left; None where unresolvedz1List[Optional[Tuple[float, float, float, float]]] source_boxesNziUnion of the located source boxes (x1, y1, x2, y2) normalized 0..1, top-left origin; None if none locatedr+rArnonezcell_address: every reported source address resolved and text-validated; partial: some resolved (the union box under-covers the combined cell); none: no address reported or none resolvedz*Literal['cell_address', 'partial', 'none'] grounded_byr)r"r#r$r%rsr rrcrrr[rrrrrr&rr rrrs $efFCf  *C'deCetAlmGYm# VL)GLrGLC8=8C 4?D (?K;r rc<[U5RSUSU3$)zThe deterministic table ID: '-p-t'. Built purely from the table's position, so the same document yields the same IDs on every run. A split sub-table appends '-s' to its parent's address at the split site. z-pz-t)rstem)rr@ordinals r table_addressrs&6l 4&7) 44r cU(dg[R"[R"U5R 55R 5SS$)z9First 8 hex characters of sha256 over the raw value grid.r*N)hashlibsha256jsondumpsencode hexdigest)cellss rgrid_fingerprintrs9  >>$**U+224 5 ? ? A"1 EEr c*^^SUU4Sjjn/n[U5HpupgU(aU[U5:aXO/nUR[U5V V s/sH'up[X"U [U5:aXOS5S9PM) sn n 5 Mr U$s sn n f)aPair a Camelot cell grid with its per-cell geometry into GroundedCells. `cell_boxes` arrive in Camelot's frame (PDF points, bottom-left origin) and are converted here to normalized 0..1 top-left, so the grid serializes in the same frame as every other box on the table. Shaped exactly like `cells`. Any position `cell_boxes` does not cover (or a None `cell_boxes`) yields a cell with no box, so the result is safe to build from a candidate whose geometry was not captured. c^>Ub TS::dTS::agUupp4UT- TU- T- UT- TU- T- 4$)Nrr)rx1y1x2y2page_hpage_ws rnormgrounded_grid..normsJ ;&A+1V frkV3R&[6B;RXBXYYr N)rYr)rrAr!rA)rlenrr) r cell_boxesrrrgridirbox_rowjtxts `` r grounded_gridrs ZZ &(DE"#-!c*o2E*-2 (n ,FA#4a#g,>N TX+YZ,  # K  s.B )rzSequence['ExtractedTable']r!zList[CellFlag])rz str | Pathr@r?rr?r!r.)rList[List[str]]r!r.) rrrzAOptional[List[List[Optional[Tuple[float, float, float, float]]]]]rrPrrPr!zList[List[GroundedCell]]))rs __future__rrrpathlibrtypingrrrrr r r pydanticr r rquber.agents.classifierrquber.agents.llm_clientrrr(rfrur~rrrcrAUTHORIZED_LABELSrrrrrrrrrr rrs$# XXX664/MXMM zYzzJyJZ2'I'T  * ! k  'O C  &R 1  ^ D  P G  !K >   e    W J   "a :  $A .  'L .  "O (   N .   !V 7  g_1-_J>?E/DEE&9&Ry./d1I1h5F Q     r