>jOhSrSSKrSSKrSSKrSSKJr SSKJr SSKJrJ r J r J r J r SSK r SSKJr SSKJrJr \(aSSKJrJr SS KJr SS KJr SS KJr SS KJr SS KJr SSK J!r! SSK"J#r#J$r$ SSK%J&r& SSS\ \'4Sjr("SS\5r)"SS\5r*"SS\5r+"SS5r,g)z Table Inference with LLM Uses PydanticAI with various model providers to analyze tables and generate meaningful titles and descriptions based on document context. N)datetime)Path) TYPE_CHECKINGAnyDictListOptional)logger) BaseModelField)DoclingDocumentNodeItem) AgentFactory) ModelProvider)HeaderConsolidator) TableMetadata)parser_for_preset) load_prompt)Settings get_settings) setup_loggingitemrreturncJ[USS5nU(dgUSR$)aThe 1-indexed page an item sits on, taken from its docling provenance. docling records an item's location in `item.prov` (a list of provenance spans, each with a `page_no`); the page lives there, not on a `_page` attribute. An item may carry several spans when it straddles a page break; the first span's page is used. Structural group nodes have no provenance and return None. `prov` is read via getattr because the base node type does not declare it (only laid-out items do).provNr)getattrpage_no)rrs >/home/mande/repo/quber/src/quber/processors/table_inference.pyprovenance_pager s& 4 &D  7??c\rSrSr%Sr\"SS9r\\S'\"SS9r \\S'\"SS9r \ \\S '\"S S S 9r \ \\S '\"SS9r\\S'\"S SS 9r\ \\S'\"S SS 9r\ \\S'\"SS9r\\S'Srg ) TableContext/z!Input context for table analysis.z#3-4 sentence document-level summary descriptionexecutive_summaryz#Title generated by procedural rulesprocedural_titlezIndividual headers foundheadersNzText that describes the table)defaultr%descriptive_textzFirst few rows of the table table_previewzText before the tablepreceding_textz-Extended page-level context including headers page_contextzPage where table appears page_number)__name__ __module__ __qualname____firstlineno____doc__r r&str__annotations__r'r(rr*r r+r,r-r.int__static_attributes__r/r rr"r"/s+"/TUsU!.STcT+EFGT#YF&+DFe&fhsmf+HIM3I$)$D[$\NHSM\"'"Q#L(3-)CDKDr r"cF\rSrSr%Sr\"SS9r\\S'\"SS9r \\S'Sr g ) TableInfo>z'LLM-inferred information about a table.z"Clear, concise title for the tabler$titlezJFunctional description of what the table contains and how it might be usedr%r/N) r0r1r2r3r4r r<r5r6r%r8r/r rr:r:>s)1#GHE3H`Kr r:c0\rSrSr%Sr\"SS9r\\S'Sr g)ExecutiveSummaryGz Executive summary of a document.z.3-4 sentence executive summary of the documentr$summaryr/N) r0r1r2r3r4r r@r5r6r8r/r rr>r>Gs*%UVGSVr r>c\rSrSrSrS%S\\S\\S\\S\4Sjjr S&S \ S \\S \\S \ \\ 44S jjr S'S\\ S\\ \\4S \\S\\S \\ \\ 44 SjjrS\S \4SjrS(S\\S\S \4SjjrS)SSS\S\S \4SjjrSSS \4SjrSSS \ \\ 44SjrS \ \\ 4S!\4S"jrS \ \\ 4S!\4S#jrS$rg)*TableInferenceProcessorMz8Process tables with LLM inference for enhanced metadata.NprovidermodelsettingspresetcU=(d [5UlXlX lX@l[ SURR RURRS9 [SS5n[URR RS9Ul URR[UUUS9Ul[R "SU=(d SS U=(d S 35 g ) a Initialize the processor with specified configuration. Args: provider: Model provider to use (auto-detected if not specified) model: Specific model to use (uses default if not specified) settings: Settings object (uses the cached get_settings() if not provided). The optional-injection seam lets callers/tests pass an overridden Settings (e.g. CLI flags). preset: DoclingParser preset to use ("tuned-financial" canonical default, or "legacy" for the pre-QUE-218 configuration). ztable-inference) service_nameenable_logfire log_leveltable_inferencesystem)rJ output_type system_promptrDrEz3Initialized TableInferenceProcessor with provider: autoz , model: r)N)rrFrDrErGrobsrJrKrr agent_factory create_agentr:agentr debug)selfrDrErFrGrPs r__init__ TableInferenceProcessor.__init__Ps&!2LN     *==,,;;mm-- $$5x@ *9J9J9Y9YZ''44!' 5    A(BTfAUU^_d_qhq^r s r metadatar-r&rc # [U=(d SUR=(d SUR=(d /URUR=(d SUR UUR S9nURU5nURRR(ac[R"SURUR S9 URRU5IShvN nUR nSSS5 O/URRU5IShvN nUR n["R$"SUR35 URUR URURURUR WR,UR.URUR0UR2UR4S.S. $N!,(df  N=fN![&a^n["R("S URS U35 [+UR=(d S URS -3S S9nSnAGNSnAff=f7f)a Analyze a single table with LLM inference. Args: metadata: Table metadata from enhanced parser page_context: Extended context from the page executive_summary: Document-level executive summary Returns: Dictionary with both procedural and LLM-inferred information zNo document summary availablezUntitled Table)r&r'r(r*r+r,r-r. analyze_table)table_idpageNzSuccessfully analyzed table zLLM inference failed for table z: zTable zUnable to generate description)r<r%)rowscols first_cell) r^r_r'r(r*r, llm_titlellm_descriptiontable_markdownrZ)r"consolidated_headerr(r*rfr,r. format_promptrFrRrJlogfirespan table_indexrUrunoutputr rV Exceptionerrorr:r<r%rarbfirst_cell_content) rWrZr-r&contextpromptresultllm_infoes rr]%TableInferenceProcessor.analyze_tables$/R3R%99M=M$$*%66"117R#22% ,,   ''0F}}  //\\/H> 9F%}}Hml $zz~~f55!== LL78L8L7MN O!,,(( ( < <'' ( 9 9&55!'33&55  &99  !:ml6   LL:8;O;O:PPRSTRUV W 22Yx?S?SVW?W>X6Y<H siA1I(4A G=G*3G(4G*'G=+G;,3G=B I((G** G84G== I%AI I( I%%I( metadata_list page_contextsmax_concurrentc^^^^ # U=(d TRRnT=(d 0m[R"U5m S[4UUUU 4SjjnUVs/sH oe"U5PM nn[ R "S[U5US9 [R"U6IShvN nSSS5 U$s snfN!,(df  W$=f7f)ak Analyze multiple tables concurrently with rate limiting. Args: metadata_list: List of table metadata to analyze page_contexts: Optional dict mapping table indices to page contexts executive_summary: Document-level executive summary max_concurrent: Maximum concurrent analyses (uses config default if None) Returns: List of analysis results rZc># TIShvN TRUR5nTRXT5IShvN sSSS5IShvN $NJNN !,IShvN (df  g=f7f)N)getrkr])rZr-r&rxrW semaphores ranalyze_with_limitHTableInferenceProcessor.analyze_tables_batch..analyze_with_limitsL y,001E1EF !//HYZZ!yyZ!yyysUA5AA51AAA A5AA5AA5A2!A$ " A2.A5analyze_tables_batch) total_tablesryN) rFmax_concurrent_tablesasyncio Semaphorerrirjlengather) rWrwrxr&ryr~rZtasksresultsr}s ` `` @rr,TableInferenceProcessor.analyze_tables_batchs&(N4==+N+N%+ %%n5  [} [ [ ?LLm(#H-mL \\ "]1CTb $NNE22G M 3  s<AC B4-!C B;%B9&B;*C 9B;; C C rqc&SURSURSUR=(d SSURSUR(aSR UR5OSS UR =(d SS URS 3$) z-Format the context as a prompt for the agent.z Analyze this table from page z&: === DOCUMENT EXECUTIVE SUMMARY === z === EXTENDED PAGE CONTEXT === z"No extended page context availablez4 === IMMEDIATE TABLE CONTEXT === Procedural Title: z Headers Found: , Nonez= Preceding Text (chronological order, furthest to closest): z Note: The LAST element in preceding text may be a footnote from the previous table. [Table] markers indicate preceding tables. === TABLE DATA (may be truncated at page boundary) === z Based on ALL context levels (document summary, page context, procedural headers, and table data), extract the most appropriate title and provide a functional description for what this COMPLETE table represents. )r.r&r-r'r(joinr,r+)rWrqs rrh%TableInferenceProcessor.format_prompts%1123  ==>?++,-.5oo '//*6JK !6"#  ) r markdownmax_rowscU(dgUR5RS5nUS[US-[U55nSR U5$)z$Get preview of table (first N rows).zNo table content available N)stripsplitminrr)rWrrlines preview_liness rget_table_preview)TableInferenceProcessor.get_table_preview sJ/ &&t,=HqL#e* => yy''r documentr r. char_limitcl/nSn/nUR5Hupx[U5U:waM[USS5=(d SR5n U (dMC[USS5n U bUR SU SU 35 OUR U 5 U[ U 5- nXS:dM O XSS-:aUS :aUS - n /n UR5HTupx[U5U :waM[USS5=(d SR5n U (dMCU R U 5 MV U (a?S R U S S5n [ U 5S :aS U SS-n UR SU -5 U(a&UR SS R USS5-5 U(aSR U5$S$)aH Extract extended context from the page containing the table. Args: document: The Docling document object page_number: The page number (1-indexed) where the table appears char_limit: Maximum characters to extract for context Returns: String containing page-level context including headers and surrounding text rtextr\levelNz [Header Lz] gffffff?r`ri...i z=== Previous Page Context === z=== Current Page Content === z zNo page context available) iterate_itemsrrrappendrr)rWrr.r context_parts char_countcurrent_page_textr_levelrr prev_pageprev_page_text prev_texts rextract_page_context,TableInferenceProcessor.extract_page_contexts$&  (*$224LDt$ 3D&"-3::>@4"))$/ !9 IInRS&9: y>C' % $%(8 8I$$%F%RS   0499=NsPR=S3TT .;v{{=)[@[[r cp# UR5nURR[SURUR S9nSUS3nUR RR(aO[R"S5 URU5IShvN nURRsSSS5 $URU5IShvN nURR$NR!,(df  g=fN-![a#n[R "SU35 SnAgSnAff=f7f) z Generate a 3-4 sentence executive summary of the entire document. Args: document: The Docling document object Returns: Executive summary string zCYou are analyzing business documents to create executive summaries.rNa5 Analyze this complete document and provide a 3-4 sentence executive summary. The summary should describe: 1. What this document is (type of document, purpose) 2. Who it is about (company, organization, or subject) 3. What are the major points being made (key metrics, themes, or findings) === DOCUMENT === zg Provide a concise executive summary that captures the essential nature and content of this document. generate_executive_summaryNz&Failed to generate executive summary: z#Unable to generate document summary)export_to_markdownrSrTr>rDrErFrRrJrirjrlrmr@rnr ro)rWrdocument_markdown summary_agentrrrsrus rr2TableInferenceProcessor.generate_executive_summaryRs%779**77(_]]** 8    9}}  //\\">?#0#4#4V#<)?@A R(^E LL;uZ'81'<&=WU6]OSTU V LL7 8 LL7>P8Q8[U[7\] ^yy## 9$))E)DT:U9VWXyy+,, 7>P8QRVSV8W7XX[\] LL. / LL={);(<= > LL.u5F/G.HI J LL* + LLuZ0895CTU[C\B]]ef Z $$\22 1% 2CL2QRUSU2V1WWZ[\yy)** 34 %&6 7 = =d CBQ G )DLL&*u-.44T:;a?LL!LM LL !C$F+sW 5 GGDIIe$ %6 )+786 5s !M M)rUrSrErGrDrF)NNNztuned-financial)NN)NNN) )i)r0r1r2r3r4r rr5rrXrrrr]rr7rr"rhrrrrrrr8r/r rrBrBMsB-1#'+' / =)/ }/ 8$ /  / h'++/ B B smB $C= B c3h B N37+/(, &M*& S#X/&$C= & ! & d38n  &P\c8((3-(3(PS(PT>\)>\8;>\IL>\ >\@/99J/9s/9b? /G? DQTVYQYN? B>d38n>3> 39c3h39c39r rB)-r4rrrrpathlibrtypingrrrrr rilogurur pydanticr r docling_core.types.doc.documentr r quber.agentsrquber.agents.factoryrquber.core.consolidationrquber.core.modelsrquber.core.parsersr quber.promptsrquber.settingsrr quber.utilsrr7rr"r:r>rBr/r rrs ;;%I%.7+0%1% * #  E9 E WyW p9p9r