j0|tSrSSKJr SSKrSSKJr SSKJrJrJ r J r SSK J r J r SSKJrJrJrJrJrJrJrJrJr SSKJr SS KJr SS KJr SS KJr SS K J!r! SS K"J#r# Sr$Sr%\ \&\&\&\&4r'\ \&\&\&\&4r(S)Sjr)S*Sjr*\RV\RX\RZ\R\1r/S+Sjr0Sr1Sr2\Rf"S5r4S,Sjr5S-Sjr6S.Sjr7S/Sjr8S0Sjr9S1Sjr:S2Sjr;S3Sjr<\RV\RX\RZ\Rz1r>S4Sjr?S5Sjr@S6S jrAS7S!jrBS8S"jrCS9S#jrDS:S$jrES;S%jrFSS'jjrHS?S(jrIg)@u#Build the unified document: move Camelot bodies into the docling spine. The matcher classified every region; this pass acts on those records with docling-core's native mutation API, so no custom serializer is needed: - `replace` — rebuild each matched docling table's body from the Camelot cells, in place, and attach the SoM title/subtitle/units/footnotes. The spine element and its reading-order position are untouched. - `docling_miss` — insert the Camelot table into the spine right after the nearest element above it on the page (before the page's first element when the table sits at the top), so reading order is preserved. - `docling_undercount` — delete the docling table(s) in the region and insert the finer-grained Camelot tables in their place. - `image_table` — no graft. docling already holds the OCR'd table. The region still takes part in the region-text dedup below, so docling text items inside it that the SoM table's text accounts for are deleted. - `chart` — no graft. docling already holds the picture. The matched picture's meta records that Set-of-Mark read it, under `READ_BY_FIELD`. - `som_miss` — no change, surfaced as an error. A table docling found but SoM did not is never synthesized from docling. The source document is never mutated: `build_unified_document` deep-copies it and mutates the copy. Deleting and inserting tables renumbers every later table's `self_ref`, so after all mutation the pass rewrites each match's `docling_table_refs` to the grafted tables' final positions. The matches ship in the fusion report beside the unified document, and a consumer resolving a ref against that document must land on the table the match is about — a stale ref silently hands it a neighboring table's identity (its section heading, its position in the flow). ) annotationsN)Counter)DictListOptionalTuple) BoundingBox CoordOrigin) DocItemDoclingDocument FloatingMetaNodeItem PictureMetaProvenanceItem TableCell TableData TableItem) DocItemLabel)logger)ExtractedTable) markdown_rows) RegionMatch) printed_keyquber__read_byz set-of-markc URSS9nURVs0sHoURU_M nn/n/nUGHn U RV s/sHoUR U 54PM n n U R S:Xa![ XFXU5 URX45 MaU R S:XaF[XIX5upURU 5 URXVs/sHnSU4PM sn45 MU R S:XaH[XFXU5upURU 5 URXVs/sHnSU4PM sn45 GMU R S:Xa`URSU RS [U R5S [U R5S 35 URX45 GMU R S :XaAURSU RS U RS35 URX45 GMU R S:Xa![XIU5 URX45 GMURX45 GM [XBX5 [!XH5 XG4$s snfs sn fs snfs snf)aGraft the Camelot bodies into a clone of `document` per the matches. Returns the unified document and a list of error strings: one per `som_miss` region (a table docling found that SoM did not match), one per `som_merged` region the split pass left unresolved, and one per `docling_miss` or `docling_undercount` SoM table with no Camelot body, which stays out of the unified document. The input document is not mutated. T)deepreplace docling_missdocling_undercount som_mergedpage z$: som_merged not resolved by split (z SoM vs z( docling); docling tables left untouchedsom_missz: docling table z< has no SoM match (SoM/Camelot missed a table docling found)chart) model_copytablesself_refdocling_table_refsgetkind_apply_replaceappend _apply_insertextend_apply_undercountpagelen som_indices _mark_vetted_dedup_table_region_text_rewrite_report_refs)document som_tablesmatches page_dimsunifiedt tables_by_referrorsrealizedmatchr survivinginsertederrss 5/home/mande/repo/quber/src/quber/core/fusion/graft.pybuild_unified_documentrEFsX!!t!,G,3NN;NqZZ]NM;FQSH8=8P8PQ8P1**1-.8P Q :: " 75i P OOU. / ZZ> )*7:QNH MM$  OOUh$?hb!Wh$?@ A ZZ/ /.wuZcdNH MM$  OOUh$?hb!Wh$?@ A ZZ< ' MM |$))*+8C8P8P4Q3RS00  OOU. / ZZ: % MM |#3E4L4L3MN==  OOU. / ZZ7 "  4 OOU. / OOU. /ILWzE+ ?i<R%@%@sI.I3I8 2I= c:[UR5VVs0sHup#[U5SU3_M nnnUHYupV/nUHFupU bUR[U 55OSn U c U(dM0UR U bU OU5 MH XulM[ gs snnf)uPoint every match's `docling_table_refs` at the unified document. Final refs are read off each tracked object's position in the tables array — the same positions serialization writes — so the rewrite cannot disagree with the document on disk. A ref that never resolved to a table is kept verbatim rather than dropped: it still names what the matcher saw, and dropping it would hide the mismatch. z #/tables/N) enumerater&idr)r,r() r:r>ir; final_refr?itemsrefsoriginalitemrefs rDr5r5s5>gnn4MN4MDAA)A3'4MIN #NH-1-=)--4)4C{8 KKsH = $ $( !OsBc lURVs/sH&o2UR(dMX#RPM( nnURVs0sHoURU_M nnURHnUR U5nUcMUR =(d [5n [[U [S5=(d /5n U R[US.5 U R[U 0S9Ul[R"SU[SR!U55 M gs snfs snf)aRecord on a picture that the table engine has already read its region. A `chart` match is a table the engine extracted from the text layer whose region overlaps a picture the parse detected. That is routinely a real table printed beside a chart, both inside the one region the engine bounded: the engine read the table correctly and drew its box around the pair. Nothing about the document changes. What is recorded is which reader has read the region and what it produced, so a later reader leaves it alone instead of stating the same table twice. The alternative is for that later step to recompute the overlap against an artifact it is not handed, which is the same judgement made again from less. The tag names the reader rather than the outcome. A region already read is the general condition; which tool did the reading is what a caller needs to weigh it, and another tool added later says so the same way. N)readerproducedupdatez"Fusion: picture {} read by {} ({})z, )r2table_idpicturesr'docling_picture_refsr)metarlistgetattr READ_BY_FIELDr, SET_OF_MARKr%rinfojoin) r6r?r7rIrRpby_refrOpicturebaseread_bys rDr3r3s$160A0A\0A1PQ]E[E[& &&0AH\%-%6%6 7%6jj!m%6F 7))**S/ ? ||,{}wt]D9?R@+8DE}g.FG  8#{DIIV^L_`*] 7sD,D, D1g\(\?z [a-z0-9]+c 0nUHnURS;aMURHnX&nURcMURURS5upURupp[ X5U-[ X5U -[ X5U-[ X5U -4nURUR/5RX45 M M U(dg0n/nURGHAnUR[;dUR(dM+URSRnURU5nU(dM^URUS5unn URSRnUR[ R":Xa/UR$UR&UR(UR*4nO4UR$U UR&- UR(U UR*- 4nUVVs/sHup[-UU5(dMUPM nnnU(dGM@SnUHXnUR[/U55nUc[1U5=nU[/U5'[3UR4U5(dMVSn O U(aURU5 GM[6R8"SUURR:UR<USR>=(d USR@SRCUR4RE5SS 55 GMD U(aURGUS 9 ggs snnf) uDrop docling body text the grafted tables now represent. docling emits a table's cell text — and its title/units/footnote lines — both inside the TableItem and as loose TextItems in the reading-order flow, so the unified output would show each fused table once as a clean table and again as scattered fragments. For every `replace`, `docling_miss`, `docling_undercount` and `image_table` region, consider the docling TextItems (text/paragraph/caption/footnote) whose center falls inside the SoM region, and delete each one whose words the SoM table's own text accounts for (its cells, title, subtitle, units, and footnotes); the grafted body plus the structured caption/footnotes is then the single representation. An `image_table` region grafts no Camelot body and attaches no SoM caption or footnotes, so a docling text item there that the SoM text accounts for is deleted with nothing written in its place. `chart` and `som_merged` regions are not deduplicated. Position alone is never grounds for deletion. The locator's region is the vision model's box, and it reaches past the table body: it is drawn to enclose the footnote lines below the grid, and it can take in what is printed under those too. Text inside the box that the table does not carry is not table text; it is the box being too big. A paragraph stating what share of a loan book floats, a sentence introducing the next table, a footnote whose text continues on the next page and so exceeds what the table carries: each stays in the document as the text it is. The separation is logged. Section headers and page headers/footers are never removed, so the spine survives. The attribution captions/footnotes added during the graft carry no provenance box, so they are never matched here. )rrr image_tableNg @g@rFTzWFusion: page {}: {} {} lies in the region of table {} but is not table text; kept: {!r}  node_items)$r*r2 som_regionr)r0minmax setdefaultr,textslabel_REDUNDANT_LABELSprovpage_nobbox coord_originr TOPLEFTlr;r@b _center_inrH_table_text_bag _covered_bytextrr]valuer'rUtitler^split delete_items)r:r8r7r9regions_by_pager?rIsomwidthheightx1y1x2y2regionbagsvictimsrNrtregions_rubox enclosingcoveredbags rDr4r4sHDFO ::] ] ""A-C~~%%MM#((NCME ^^NBB"kE)3r;+?RuATVYZ\VadjVjkF  & &sxx 4 ; ;VM J# $&D G  ::. .dii ))A,&&!%%g. MM'>: 6yy|      3 3 3664664664662C666DFF?DFFFTVVODC,3OG[Vz#v7NSG O C((2c7#C{&5c&::d2c7m499c**   NN4 KKi    ! %%;1););*3B/0  7H0/Ps =M M cURURUR/nURH*nUR UR UR /5 M, [UR5HnUR U5 M [[RSRSU55R555$)z|Every word the grafted table carries, with multiplicity: its cells, title, subtitle, units, and footnote markers and bodies.rhc36# UHo(dM Uv M g7fN).0r_s rD "_table_text_bag..>s-DA!aas  )rsubtitleunits footnotesr.markerr}rmarkdownr _TOKEN_REfindallr^lower)tablepartsfnrows rDr{r{6s[[%..%++ 6Eoo bii)*U^^, S- 9$$SXX-D-D%D%J%J%LM NNc*[RUR55nU(dgUR5nSnUHnX5S:dM X5==S-ss'US- nM [ U5U- nU[ :*=(d U[ U5- [ :$)zWhether the table's words account for `text`: at most one token of it is missing from the bag, or all but a rendering-noise fraction are present.Trrd)rrrcopyr1_UNMATCHED_TOLERANCE _COVERAGE_MIN)r}rtokens remainingmatchedtok unmatcheds rDr|r|As  tzz| ,F  IG >A  Na N qLGF g%I , , V#f+0E0VVrc"USUS-S- USUS-S- p2[USUS5[USUS5pT[USUS5[USUS5pvXBs=:*=(a U:*Os =(a XSs=:*=(a U:*$s $)Nrrd)rmrn)rrcxcyx0y0rrs rDrzrzQs!fs1vo "SVc!f_$9 F1I &F1Ivay(A F1I &F1Ivay(A >>r> ,bnn"n,n,rc[X#U5HgupVURU5nUcMX5nUR=(d SR5n U (dMK[ U 5Ul[ XX5 Mi g)zHRebuild each matched docling table's body from its paired Camelot table.Nr)_pair_by_vertical_orderr)rstripmarkdown_to_table_datadataattach_attribution) r:r<r?r7r9som_idxrO table_itemrrs rDr+r+Xsn0=Q "&&s+   !LL&B--/ 0: 7?Rrc/n/nURHnX&nUR=(d SR5nU(d/URSURSUR <S35 M`[ U5n [Xs5n [XU5upU cURXS9n OURXXS9n [X Xs5 URU 5 M XE4$)a3Insert a Camelot table docling missed, at its reading-order position. Returns the inserted tables and an error for any table that has no Camelot body: a table SoM located but neither docling nor Camelot could read (a scanned table on a page docling did not detect). It is surfaced, never silently dropped. rr"zE: SoM located a table docling missed but Camelot read no body (title=z$); table not in the unified documentrrssiblingrrsafter) r2rrr,r0rr _som_provreading_order_anchor add_table insert_tabler)r:r?r7r9r&r=rrrrrsrrrBs rDr-r-ns!FF$$!LL&B--/ MMz""yym+OQ  %h/(-gIF ?((d(>H++GT+_H7c= h#%$ >rc^/n/nURVs/sHowU;dM XPM nn[U4SjUR5SSS9n Sn Sn U b [X XHS9upU(aUR US9 Sn [ URU4SjS 9Hn TU nUR =(d S R5nU(d/URS URS UR<S 35 Ma[U5n[X5nU bURU UUSS9n O&U bURU UUU S9n OURUUS9n [X X5 URU 5 M XV4$s snf)aDelete the docling table(s) in the region and insert the finer Camelot tables. docling under-counted (dropped or merged a complex table); Camelot's finer count is trusted. The Camelot tables are inserted after the spine element above the region, in vertical order. Returns the inserted tables and an error for any Camelot table with no body, so a dropped sub-table is surfaced rather than lost. c3.># UH nTUv M g7frr)rrIr7s rDr$_apply_undercount..s2 11A 1sc[U5$r _vertical_key)r;s rD#_apply_undercount..s mA&rN)keydefaultT)excluderjc >[TU5$rrrIr7s rDrrs=TU;Wrrrr"zD: a sub-table of a docling-merged region had no Camelot body (title=z); not in the unified documentrr)r(rmr2rrsortedrrr,r0rrrrrr)r:r<r?r7r9r&r=r@doomed anchor_somrrlastrrrrrss ` rDr/r/s!FF8=8P8Pg8P1YfTf.m.8PFg2 1 12 &J #'G E-g9] / $D%++1WX!LL&B--/ MMz"))&DF  %h/(  ''4dRV'WD  ''dUZ'[D$$$T$:D7#9 d%Y& >Ehs E%E%cUR(aHUR=(d [5nURURURS.S9Ul[ XU5 /n[ X5H@nUR[RUS9nURUR55 MB U(aXQl UnURHn U R(a)U RSU R3R!5O U RnUR!5(dM`UR#[R$XhSS9nM g)uCarry the SoM identity, cell geometry, title, subtitle, units (captions) and footnotes onto a table. The SoM table's `table_id` and `content_fingerprint` bind onto the docling item through `meta` (docling's sanctioned extension point — the meta model allows extra fields, so they serialize with the document), and every grafted cell carries its measured box where the provenance record can pin one. SoM is the vetted, vision-grounded side, so its attribution takes precedence over whatever docling attached: - the sentence that introduces the table, its printed name and its units become CAPTION nodes referenced from the table (top-to-bottom order; see `caption_lines`). The table serializer emits captions inline, so they render above the table in markdown/HTML. - each footnote is inserted as a FOOTNOTE node in the body right after the table, in order. docling's markdown serializer does NOT emit a table's `footnotes` refs, so a referenced footnote would round-trip in JSON but vanish from the rendered markdown; a body node after the table renders as a paragraph below it and keeps the reading order. Both kinds carry no provenance box, so the region-text dedup that strips docling's loose copies never matches them. )quber__table_idquber__content_fingerprintrS)rqr}rhT)rqr}rrN)rUrXr r%content_fingerprint_graft_cell_geometry caption_linesadd_textrCAPTIONr,get_refcaptionsrrr}r insert_textFOOTNOTE) r:rrr9rb caption_refsr}captionanchornotes rDrrs: ||0,.//#&<<.1.E.E*  )4Lg+"")=)=D"IGOO-.,*!F 8<{{$++a {+113 zz|| $$<+@+@tcg$h rcP[XR5n/nUR=(d SR5nU(a=[ U5U;a.UR [ XRU5=(d U5 UR=(d SR5nU(a[ U5U;aSnU(a3U(a[ U5[ US5;dUR U5 UR(aHURR5(a)UR URR55 UR(aHURR5(a)UR URR55 U$)aThe header lines a table carries, top to bottom: the sentence that introduces it, its printed name, its units. The introducing sentence is the docling paragraph on the table's page that contains the caption the vetting agent copied. Two tables printed side by side under one sentence then carry the same whole sentence, whichever words each copy kept, and a bold lead-in the copy left out comes back with the paragraph. When no such paragraph exists, the copied caption stands. The fusion graft calls this before its region dedup runs, so docling's copy of the sentence is still in the document at that point. A name that repeats the sentence is not written twice. Page furniture is never a header line. A running header or footer the parse labeled as such ("Table of Contents", the statement banner) is printed on the page, so the printed-text check upstream lets it through when the agent copies it; the parse's own label is what rules it out. rr) _furniture_keysr0rrrr,_paragraph_containingrrr)r:r furniturelinesrrs rDrr s$ 2IE{{ b'')G;w'y8 *7HHgFQ'R YY_" # # %E U#y0 e E 2k%(6K K U || **,, S\\'')* yySYY__&& SYY__&' Lrcb[5nURHnUR[R[R 4;dM3UR (dMFUR SRU:XdMe[UR5nU(dMURU5 M U$)zVThe printed keys of the page's running headers and footers, as the parse labeled them.r) setrprqr PAGE_HEADER PAGE_FOOTERrsrtrr}add)r:r0keysrNrs rDrr-sxUD  JJ<33\5M5MN N  ! $$,dii(Cs  KrcP[U5nURHnUR[;d.UR(aURSR U:waMGU[UR 5;dMbSRUR R55s $ g)zThe text of the first docling item on `page` whose printed words contain `caption`, whitespace collapsed; None when no item does. Items without a page box are skipped, which excludes the caption nodes the graft adds.rrhN) rrprq_CAPTION_SOURCE_LABELSrsrtr}r^r)r:r0rrrNs rDrr<sy g C  ::3 3499 RS H\H\`dHd  +dii( (88DIIOO-. .  rcX[U5nU(d [/SSS9$[SU55n[U5n/n[ U5HVupVUS/U[U5- --n[ U5H-upUR [ U UUS-UUS-SSUS:HS95 M/ MX [XCUS9$)aParse a pipe-delimited markdown table into a docling `TableData` grid. The first content row is treated as the column header. The markdown separator row (`| --- | --- |`) is dropped. Ragged rows are padded to the widest row so the grid stays rectangular. r) table_cellsnum_rowsnum_colsc38# UHn[U5v M g7fr)r1)rr@s rDr)markdown_to_table_data..Ts(4a3q664srrd)r}start_row_offset_idxend_row_offset_idxstart_col_offset_idxend_col_offset_idxrow_spancol_span column_header)rrrnr1rGr,r) rrowsrrrr@rpaddedcr}s rDrrIs  "D R!a@@(4((H4yH#%KD/tx#c(233 (GA   )*'(1u)*'(1u#$6  )" ( SSrc URnU(dgURURS5upEURRHnUR UR pU[U5:a"U[X75:aX7UROSn U cMRU upp[[X5U-[X5U-S[X5- U-S[X5- U-[RS9UlM g)uCarry the already-resolved cell geometry onto the grafted TableCells. `corrected_grid` is the complete cell-level view the grounding stage produced — same markdown, same row/col addressing — so this is pure carriage: read each cell's box and convert it from the normalized top-left frame to docling's bottom-left points, the same convention `_som_prov` uses for the table-level box. No matching or reconciliation happens here; a cell the grounding stage could not pin arrives with box=None and stays without one. Nrg?rxr@r;ryrv)corrected_gridr)r0rrrrr1rr rmrnr BOTTOMLEFTru)rrr9gridrrcellr@rrrrrrs rDrris   D MM#((N;ME++(($*C*C1 !CI !c$'l2Bdgajnn ; "kE!"kE!S[ F*S[ F*$//   ,rc^^[URU4SjS9n[U4SjUR5U4SjS9n[[ X4SS95$)zEPair SoM tables to docling tables in a region by top-to-bottom order.c >[TU5$rrrs rDr)_pair_by_vertical_order..s=A3Orrc36># UHoT;dM Uv M g7frr)rr@r<s rDr*_pair_by_vertical_order..sC,q]0B,s  c >[TU5$r) _table_top)r@r<s rDrrsjq!12rF)strict)rr2r(rYzip)r?r7r<somsrLs `` rDrrsG %##)O PD CE,,C 2 D Du- ..rcURnUb[USUS5$URb'[URSURS5*$g)Nrdr)rlrmrurn)rrs rDrrsU   F 6!9fQi(( zzEJJqM5::a=111 rcUR(dgURSRnUR[R:Xa UR $[ UR UR5*$)Nrr)rsrurvr rwr;rnry)rrus rDrrsT :: ::a=  D K///vv   rc 4URURS5up#URnUcgUupVpx[[ XW5U-[ XW5U-S[ Xh5- U-S[ Xh5- U-[ RS9n [URU SS9$)zJA ProvenanceItem for an inserted Camelot table, box in bottom-left points.rgNrr )rr)rtrucharspan) r)r0rlr rmrnr r r) rr9rrrrrrrrus rDrrsMM#((N;ME ^^F ~NBB  b+  b+  R  & R  & ++  D #(( GGrcURURS5upEURnUcg[USUS5U-nU=(d /Vs1sHn[ U5iM n n/n UR 5Hup[ U [5(a[ U 5U ;aM+U R(aU RSOSn U bU RUR:waMl[X5nU RX45 M U (dgU RSS9 U VV s/sHupX:dM U PM nnn U(aUS S 4$U SSS 4$s snfs sn nf) anFind the spine element to anchor an inserted table to. Returns (sibling, after). The sibling is the last element above the table on its page, with after=True. When the table sits above every element on the page, returns that page's first element with after=False. Returns (None, True) when the page has no other anchorable element (caller falls back to add_table). rgN)NTrdrrc US$)Nrr)xs rDr&reading_order_anchor..sqtrrTF) r)r0rlrmrH iterate_items isinstancer rsrt_item_top_left_topr,sort)r6rr9r_widthrr table_topeexcludedon_pagerN_levelrstopaboves rDrrs9]]388^7iccoT7E > Ry$ 1:a=% %0 ?sE$ E3EcURnUR[R:Xa UR$U[ URUR 5- $r)rurvr rwr;rnry)rs page_heightrus rDr$r$sB 99D K///vv TVVTVV, ,,r) r6r r7List[ExtractedTable]r8List[RegionMatch]r9Dict[int, Tuple[float, float]]returnz!Tuple[DoclingDocument, List[str]])r:r r>z?List[Tuple[RegionMatch, List[Tuple[str, Optional[TableItem]]]]]r3None)r6r r?rr7r0r3r4) r:r r8r1r7r0r9r2r3r4)rrr3 Counter[str])r}strrr5r3bool)rBoxrr8r3r7) r:r r<Dict[str, TableItem]r?rr7r0r9r2r3r4) r:r r?rr7r0r9r2r3!Tuple[List[TableItem], List[str]]) r:r r<r9r?rr7r0r9r2r3r:) r:r rrrrr9r2r3r4)r:r rrr3z List[str])r:r r0intr3zset[str])r:r r0r;rr6r3z Optional[str])rr6r3r)rrrrr9r2r3r4)r?rr7r0r<r9r3zList[Tuple[int, str]])rrr3float)rrr3r<)rrr9r2r3zOptional[ProvenanceItem]r) r6r rrr9r2rzOptional[List[NodeItem]]r3zTuple[Optional[NodeItem], bool])rsrr/r<r3r<)J__doc__ __future__rre collectionsrtypingrrrrdocling_core.types.doc.baser r docling_core.types.doc.documentr r r rrrrrrdocling_core.types.doc.labelsrlogururquber.core.extractors.baser1quber.core.extractors.set_of_mark.merge_groundingrquber.core.fusion.modelsrquber.core.printed_textrr[r\r<NormBoxr8rEr5TEXT PARAGRAPHrrrrr3rrcompilerr4r{r|rzr+r-r/rSECTION_HEADERrrrrrrrrrrrr$rrrDrOs<@# ..@   75K0/!  ueU* + E5% &'CC$CC. C ' CL( (M( (6 aH  JJ| $ [1 [1 [1%[1. [1  [1|O W -@ @'@ @% @ . @  @,! ! !%!. ! ' !H3 3'3 3% 3 . 3 ' 3l5i 5i5i 5i. 5i  5it !H  T@  . ;Y   @ /  /$ /( / / H*)- % % % .% & % % % P-r