jrbSrSSKJr SSKJr SSKJrJrJrJ r J r SSK J r J r SSKJr SSKJrJr SSKJrJrJrJr SS KJr SS KJr SS KJr SS KJr SS K J!r!J"r"J#r# SSK$J%r%J&r&J'r' SSK(J)r)J*r* SSK+J,r,J-r-J.r. SSK/J0r0 Sr1Sr2Sr3Sr4Sr5Sr6Sr7S\1SS4S)Sjjr8S*Sjr9S+Sjr:S,Sjr;S-Sjr<S.S/S jjr=S0S!jr>S1S"jr?S2S#jr@S3S$jrAS4S%jrBS5S&jrCS6S'jrDS7S(jrEg)8a Put each figure's text onto the picture the parse already found. The placeholder is already in the document. Every picture the parse detected sits in the body in reading order carrying its page and its box, and nomination read that placeholder rather than creating it. Grafting fills it in: the picture that was always there now carries the figure's content. So a chart is not a new kind of element competing with tables for a position. It carries no cells and no per-value location, and is not shaped like a table. The text lands as the picture's description, the document model's own field for what a picture depicts, and the scan's identity lands in the picture's metadata so a picture can be traced back to the scan that read it and a model change is visible without re-reading the text. Each record is given the reference of the picture it filled, so a plotted value traces back to the element that holds it. The two counts do not always agree. The scan decides, because it looked at the page and the box it drew is already a source coordinate, while the parse is working from a layout model. So a region ends up holding exactly as many pictures as the scan returned figures: - One picture the scan read as several figures is split into one per figure. The parse draws a single region over two charts printed side by side. - Several pictures the scan read as one figure become one. The parse splits a chart pair the scan describes together. - A region already agreeing keeps the box and bitmap the parse measured. - A figure over no picture at all is surfaced as an error and never dropped, since a figure the scan read that nothing holds is a value that vanishes. Figures and pictures are grouped by overlap before anything is assigned, so the order the figures arrive in never decides an outcome. What a picture *is* remains the parse's call, not the scan's. The scan describes a wordmark as readily as a chart, and a branded deck prints one on every page, so a region the parse classified as page furniture is dropped with its records. That judgement reads a class the parse already published rather than a rule written against a description that has no fixed shape. The text the parse lifted out of a picture is removed once the scan has read it, except for items whose label the caller names in `preserve`. Those fragments are bare numbers with nothing saying what they measure, and the description now carries the same values with their series and their axes. Keeping both would put two readings of one chart in the document. This module attaches no titles or notes to a picture. One note commonly serves several charts on a page, and nothing here matches a superscript to the note it points at. The correction sweep in `quber.core.figures.correct` writes each figure's markers and page notes onto its picture's metadata under `FOOTNOTE_MARKS_FIELD` and `FOOTNOTES_FIELD`. On the dpt-3 track, `quber.core.figures.dpt3.footnotes` then resolves each marker to its note. A table read off a page image goes into the table the parse already found for it, the same way. The element stays a table; only its body changes, from the parse's own reading of the image to the scan's, along with the title, subtitle, units and footnotes read off the page with it. `graft_figures` never mutates the document it is given and returns a clone. `graft_tables` mutates the document it is given. The orchestrator runs it first, on its own copy of the parse, and then hands that copy to `graft_figures`. ) annotations)Path) CollectionDictListOptionalTuple) BoundingBox CoordOrigin)DescriptionMetaField)ImageRefProvenanceItem)DoclingDocument PictureItem PictureMetaTextItem) DocItemLabel)logger)ExtractedTable)coverage_fraction)NormBoxnorm_boxprov_box) FigureRecordPageScan ScannedTable)FURNITURE_CLASSESpicture_classes) READ_BY_FIELDattach_attributionmarkdown_to_table_data) render_regiong?g?zquber-chart-scanadequber__footnote_marksquber__footnotesHNcZ^URSS9n[U=(d 05nURR5HIupX;dM U RcMU RR U RR 4Xx'MK 0n URHNn U R(dMU RU RSR/5RU 5 MP /n /n UHmURTRS5upU RTR/5n[TRUXU5unnUH<nU RSTRSUR SUR"S35 M> UHNunn[%U5(a['TUU5 M%U R)U4S j[+UUUTXU555 MP M U Hun nm[-U UT5 M [/XmV VVs/sH un nnU PM snnn U5 Xl4$s snnn f) aNCarry each scan's figure text onto the matching picture in a clone of `document`. `page_dims` gives each page's width and height in points, used to bring the parse's picture boxes into the scan's normalized frame. Pages absent from it fall back to the document's own page sizes. `source` is the document the parse was made from. Splitting a picture moves the boundaries its stored bitmap was cut to, so each part's bitmap is cut again from the page. Without the source a split part carries no bitmap and the loss is logged. `preserve` names text labels that survive under a picture the scan has read. The scan's reading supersedes the parse's stray content fragments, but an item the parse labelled a footnote or a caption is native page text with a known role, not a reading of the figure, and a caller passing its label keeps it in the document. Returns the refined document and one error string per figure that overlaps no picture. The records in `scans` are given the reference of the picture each one filled, and a record whose region the parse called page furniture is removed from the scan. Tdeeprg @g@page z : the scan returned a figure at z, that overlaps no picture in the parse (job z)); its text is in the figure records onlyc30># UH upXT4v M g7fNr').0picturechartscans 6/home/mande/repo/quber/src/quber/core/figures/graft.py graft_figures..s&gNG&&gs) model_copydictpagesitemssizewidthheightpicturesprov setdefaultpage_noappendgetpage_regionsfiguresboxjob_id _is_furniture_discardextend_reshape_attach_discard_superseded)documentscans page_dimsmatch_fractionsourcepreserverefineddimsr@rCpictures_by_pager0errorspairsr;r< candidatesregions unmatchedr1r=rE_c_sr2s @r3 graft_figuresr^ws<!!t!,G  R D ,,.   499#8!YY__dii.>.>?DM/68## <<<  ' ' Q(?(? D K KG T$F?AEN; %))$))R8 %dllJ~^E MM {"B599+N--2\\N:ce  ") HgX&&w1 LL&.w'4QV`f&g  ")&!&%!&!Gogr2'!GR ?"Hs H&c DUVs0sH"oDR(dMURU_M$ nnURVs0sHoDRU_M nn/nURVs0sHoRU_M n nUGHn U RGHwn U RcMU RcU R b[ X XXSU5 M=U RcMLURU R5n URU R5n U bU c;URSU RSU RSU RS35 MU R=(d SR5(d/URSU RSU RS35 GM[R"SU RU RU R5 [U R5U l[!X X5 GMz GM U$s snfs snfs snf) uPut each table read off a page image into the parse table it was read for. The element stays a table. Nothing is relabelled: a balance sheet is a table whichever tool read it, and calling it an image so it could travel a picture-shaped path would bend the document to suit the code. The extracted body replaces the parse's own reading of the image, and the title, subtitle, units and footnotes the correction read off the page are attached with it, exactly as they are for a table the table engine produced. Nothing is lost by replacing: the page's raw response is stored as it came back, so what was read and by which model version stays on record. Mutates `document` in place — the caller owns the clone — and returns one error per scanned table it could not put in: one whose parse table or extraction cannot be found, one whose extraction has an empty body, and one read over a picture that cannot be resolved or produced no body. r,z%: the table read off the page image (z) has no home in the parse (z"); it is in the table records onlyz!: the scan returned a table over zK with no cells in it; the parse's own reading of the image is left standingz>Table graft: page {} replacing {} with the scan's reading ({}))table_idtablesself_refr= table_ref picture_ref_add_beside_picturerBrArCmarkdownstriprinfor!datar )rNrOrbrPtby_idby_refrWp by_picturer2scanned table_item extracteds r3 graft_tablesrss.%+ 9Fqjj]QZZ]FE 9%-__ 5_jj!m_F 5F)1):):;):A**a-):J;{{G'  (W-@-@-L#HG[ab  (G$5$56J '"2"23I!Y%6 DII;&KGL\L\K]^--4->->,??ac&&,"3355 DII;&GHYHYGZ[YY KKP !!    5Y5G5GHJO xY J;#> MK : 5.$sC7a 117r`c3<># UHn[TU5v M g7fr.) _overlaps)r/otherrFs r3r4ry+s"cFbU9S%#8#8Fbsr,z : picture z (, unclassifiedu) was read by nothing — the scan returned no figure over it and the document holds no table there; whatever it shows is not in the document)rbr>r@rBrr?rAr=rallmeta descriptiontextrhanyrcjoin) rNrPrWtables_by_pagetablerCr;r<r0classesrFs @r3unread_picturesrs,F/1Nzz zz!}$$! dN; uV, ?  % %dB / 6 6s ;!$$|| !'* sC7CCC -4\\gll>V>VGLL $ $ ) )\^ e e g g ||A&&! dN; wv. ?s"cnFXFXY]_aFb"ccc  D6G$4$4#56 '"4n56  %& Mc[USUS5[USUS5:=(a% [USUS5[USUS5:$)z'Do the two boxes share any area at all?r)minmax)oner}s r3r|r|6sZ s1vuQx 3s1vuQx#8 8 jSQq=RUXY\]^Y_afghaiUj=jjrcVURUR=(d S5nURUR=(d S5nUb+Ub(UR=(d SR 5(d-UR SUR SURS35 g[R"SUR URUR5 UR[UR5UR(aURSOSS9n [X X5 g) a(Put a table the parse filed as a picture into the document as a table. The parse detected the region and called it a picture. The scan read it and returned a grid, so the page prints a table and the document should hold one. It is added rather than swapped in, because the picture is a real element and the printed image is what the values were read from. Nothing else about it is special. The body is the corrected grid every other scanned table carries, attached the same way, on a provenance box taken from the picture the table was printed over. r`Nr,z%: the scan read a table over picture zm, which the parse holds no table for, and the grid produced no body; the values are in the table records onlyzDTable graft: page {} adding the scan's reading of {} as a table ({})r)rjr>) rBrerargrhrArCrri add_tabler!r>r ) rNr2rprorlrPrWr0rrrqs r3rfrf;s(nnW006B7G '**0b1I)+I4F4F4L"3S3S3U3U DII;CGDWDWCXY   KKN   ## #I$6$6 7 ' W\\!_$$JxYBrcl/nUHanURHNnURU5n[U[5(dM+URU;dM=UR U5 MP Mc U(dg[ R"S[U5[U55 URUS9 [U5$)uiRemove the text the parse pulled out of a picture the scan has now read. A picture the parse detected carries the fragments it managed to lift off the image as its own children — a chart's plotted labels arrive as `$0.05`, `$0.02`, `$0.06`, bare numbers with nothing saying which series or which year they belong to. Once the scan has read the figure, those fragments are the same region read worse, and keeping them puts two readings of one chart in the document with only one of them saying what the numbers mean. An item whose label is in `preserve` stays: it is native page text with a role the parse named, not a reading of the figure. This is the same replacement a table gets. A table read off a page image has its body replaced rather than doubled, for the same reason and with the same justification: the raw response is on disk, so nothing is unrecoverable. rz`Figure graft: removing {} text fragment(s) the parse lifted from {} picture(s) the scan has read node_items) childrenresolve isinstancerlabelrArrilen delete_items)rNr=rSdoomedr0childitems r3rMrMes*F%%E==*D$))djj.H d#&  KKj F  H   V, v;rc ^^^^UU4SjU5VVs/sHupVU(dMXV4PM nnn[U5VVV s0sHunupYURU_M sn nnm0n 0n [U5Hup[U R5n U cMUHup[ [ X5[ X55nUU:dM+U R UR/5RU5 U R U/5RUR5 M M [U5VV s/sHupX;dM U PM nnn UVV s0sHupYURU_M sn nm/n[5n[5nUGH:unnURU;dURU ;aM)UR1n[5nUR/nU(aUR5nU RU/5H_nUU;aM URU5 U RU/5H-nUU;dM URU5 URU5 M/ Ma U(aMUU-nUU-nUR[U4SjU5U4SjS9[[U5Vs/sHoUPM sn545 GM= UU4$s snnfs sn nnfs sn nfs sn nfs snf)uGroup figures and pictures that overlap into regions, one region per subject. Every figure is scored against every picture before anything is grouped, so the order the figures arrive in never decides an assignment. Overlap is transitive: a figure joins every picture it overlaps and a picture joins every figure that overlaps it, and the connected result is one region. A region can then hold any mix — one to one, one picture the scan read as several figures, several pictures the scan read as one figure, or a tangle of both. Returns the regions in the pictures' document order, each region's pictures in document order and its figures in reading order, plus the figures that overlap no picture past the threshold. c3@># UHo[UTT54v M g7fr.)r)r/rnr<r;s r3r4_regions..s$Yj!UF)C%Djsc3.># UH nTUv M g7fr.r')r/rrms r3r4rs04aq 4sc">TUR$r.rc)rnorders r3_regions..sajj@Qrkey) enumeratercrrFrrr?rAsetpoprBaddsorted_reading_order)rErYr;r<rQrnrFboxedi_boverunderr1cboxr0pboxcovr[rZ seen_pictures seen_charts_boxrefsidxsfrontierrefr}rmrs `` @@r3rDrDs(%Zj$Y a$Y&!]`XaX$YE a-6u-= >-=zq'1QZZ]-= >E"$D"$Eg& " < "MG'35Ft5RSCn$ 0 0"5<M /4 Bs' J<J<K' K 6K KKcSSjn[XS9$)zFFigures in the order the page prints them: down the page, then across.cr[UR5=(d Sn[US[- 5US4$)N)rrrrr)rrFroundROW_BAND)r1rFs r3r_reading_order..keys3uyy!9%9c!fx'(#a&11rr)r1rreturnzTuple[int, float])r)rErs r3rrs2 ' ##rcUVVs/sHn[U5Ho"PM M nnn[U5=(a [SU55$s snnf)uTrue when the parse called every picture in the region page furniture. A logo or an icon is the same mark on every page of a deck, and the scan describes it as readily as it describes a chart. The parse's class is the signal for what a picture is, and it is trusted here rather than reading the scan's description to guess — one is a prediction the parse already made and published, the other is a rule written against prose with no fixed shape. c32# UH o[;v M g7fr.rvrws r3r4 _is_furniture..s IA&7!7rz)rboolr)r=r0rxrs r3rHrHsF!)KW/'2JQq2JqGK = IS I IIILsA c[R"SUR[U5SR [ UVVs1sHn[ U5HoDiM M snn555 UH/nXPR;dMURRU5 M1 UR(dURS:XaSUl gggs snnf)aEDrop the records for a region the parse called page furniture. The scan read the region and was right about it, so nothing is being corrected. The record is dropped because a description of a wordmark is not document content, and keeping it would put a figure on every page of a branded deck and count it among the figures. zDFigure graft: page {} dropping {} record(s) over page furniture ({})r~rEemptyN) rrirCrrrrrEremovestatus)r2rEr=rnrxr1s r3rIrIs KKN  G  &XJXq7I!!7I!XJKL   LL LL   & <. s=Hqwwd*HsrzGFigure graft: page {} the scan read {} as {} figure(s); merging into {}r~c38# UHoRv M g7fr.rrs r3r4r)s3(Qjj(srzeFigure graft: page {} picture {} covers {} figures the scan returned separately; splitting it into {}T)siblingr>rafterr))rrdpiDEFAULT_CROP_DPIrrrrirCrrcrlist_provr>_cropinsert_picturer6rA)rNr=rEr2r;r<rRsurvivor template_metar wanted_imagereshapedrXanchorr1s r3rKrKs@{HMMM ( :(..  @PC=H==L8}q 4CL1$4H 1:&'' 8}q U II II3(3 3 L      hqrl);< 7|c(m#  # II    L L  71:tyy%@AHMBNU699gaj#>TXHN # $E"F((uii7:F% 56D )   $'222=FK f_% Lrc f[UR5nUbUc)[R"SUURUcSOS5 g[ R "[XXC5US9$![aAn[R"SUUR[U5RU5 SnAgSnAff=f)a.The part's own bitmap, cut from the page at the resolution the parse used. Returns None when there is no source to cut from or the cut fails, so a split still happens and the part carries a box with no bitmap rather than one that depicts the wrong region. Either way the loss is logged, never silent. NzIFigure graft: page {} figure {} keeps no bitmap ({}); its box is recordedzno source document to cut fromzthe scan returned no box)rrzYFigure graft: page {} figure {} keeps no bitmap; cutting it from the page failed ({}: {})) rrFrwarningchunk_idr from_pilr" Exceptiontype__name__)rRrCr1rrFexcs r3rrJs 599 C ~ W  NN06 ,D^     }V3'LRUVV  g  NN I     sA%% B0/7B++B0cjUR=(d [5nURS[UR[ S9SUR SURSURSUR[/[U[S5=(d /Q[/S.P0S 9UlURUlg) uCarry the figure's text and the scan's identity onto one picture. Both ride on `meta`. The text goes in its description field, the document model's own place for what a picture depicts, and the scan's identity goes in alongside — the meta model allows extra fields, so they serialize with the document. The picture's existing metadata, its class prediction included, is kept. The record is given this picture's reference in return. r)r created_byquber__figure_job_idquber__figure_modelquber__figure_versionquber__figure_chunk_idN)readerproduced)update)rrr6r rDESCRIPTION_AUTHORrGmodelversionrrgetattrADErcre)r0r1r2bases r3rLrLgs << (;=D?? /UZZL^_ "DKK !4:: #T\\ $enn igdM4@FBiTWegIhi  # GL ((Erc[UR5=(d SupEpg[XB-Xb-SU- U-SU- U-[RS9n[ XSS9$)zBA provenance record for one figure, its box in bottom-left points.)rr?rr)lrrkb coord_origin)rr)r@bboxcharspan)rrFr r BOTTOMLEFTr) r1rCr;r<x1y1x2y2rs r3rr~s^eii(@,@NBB  * * 8v  8v  ++  D $F CCr)rNrrOList[PageScan]rPz(Optional[Dict[int, Tuple[float, float]]]rQfloatrROptional[Path]rSCollection[DocItemLabel]rz!Tuple[DoclingDocument, List[str]]) rNrrOrrbzList[ExtractedTable]rPDict[int, Tuple[float, float]]r List[str])rNrrPr rr )rrr}rrr)rNrr2rrprrozDict[str, PictureItem]rlzDict[str, ExtractedTable]rPr rWr rNone)r')rNrr=List[PictureItem]rSr rint) rEList[FigureRecord]rYr r;rr<rrQrrzMTuple[List[Tuple[List[PictureItem], List[FigureRecord]]], List[FigureRecord]])rErrr)r=r rr)r2rrErr=r rr )rNrr=r rErr2rr;rr<rrRrrz&List[Tuple[PictureItem, FigureRecord]]) rRrrCrr1rrrrzOptional[ImageRef])r0rr1rr2rrr ) r1rrCrr;rr<rrr)F__doc__ __future__rpathlibrtypingrrrrr docling_core.types.doc.baser r "docling_core.types.doc.common.metar 'docling_core.types.doc.common.referencer rdocling_core.types.doc.documentrrrrdocling_core.types.doc.labelsrlogururquber.core.extractors.baser5quber.core.extractors.camelot.correspondence.geometryrquber.core.figures.geometryrrrquber.core.figures.modelsrrrquber.core.figures.nominaterrquber.core.fusion.graftrr r!quber.files.pdfr"MATCH_FRACTIONrrrFOOTNOTE_MARKS_FIELDFOOTNOTES_FIELDrr^rsrr|rfrMrDrrHrIrKrrLrr'rr3r$s;z#::@CL__65SCCJJJ]]) (  / %  ;?*!)+ FF F8F F  F ' F'FR<< < !<. <  <~44-44nk 'C'C 'C'C' 'C % 'C . 'C 'C 'CZ*,###'# #LC C!C C  C  C S CL$ J*JJJ J  J  J  J J,JZ:). Dr