_i2|dZddlZddlmZddlmZmZmZmZddlm Z ddl m Z ddl m Z mZdd lmZGd d Zy) z Enhanced Table Parser with Header Consolidation Parses tables from Docling documents and consolidates consecutive headers that immediately precede tables, following strict rules to avoid incorrect consolidation across content boundaries. N)datetime)AnyListOptionalTuple) InputFormat)PdfPipelineOptions)DocumentConverterPdfFormatOption) TableMetadatac eZdZdZ ddedededefdZdedee e e ffd Z d e e d edee ee effd Z d e edefdZdd e e d ededeefdZde fdZde e defdZy)EnhancedTableParserz Enhanced table parser with intelligent header consolidation. Consolidates consecutive headers of the same level that immediately precede tables, with strict rules to prevent incorrect consolidation. separator max_lookbackenable_consolidationdebugc<||_||_||_||_y)aL Initialize the enhanced table parser. Args: separator: String to join multiple headers (e.g., " ^ " or " | ") max_lookback: Maximum elements to look back for headers enable_consolidation: Whether to consolidate consecutive headers debug: Enable debug output N)rrrr)selfrrrrs 5/home/mande/repo/quber/src/quber/core/table_parser.py__init__zEnhancedTableParser.__init__s" #($8! pdf_pathreturnc t}d|_d|_d|j_t t jt|i}|j|}|j}g}d}t|j}|D cgc]\} } |  } } } t| D cic]\} } t| | }} } |jD]}t!|dr>|j"r2t!|j"ddr|j"dj$nd}|j't|d }g}g}|dkDr|j)| |\}}d }|j*r|r|j-|}d}d}d }t!|d r|j.r|j.j0}|j.j2}|dkDr|dkDrt!|j.d rx|j.j4rbt7|j.j4dkDr|j.j4dng}|r#t7|dkDr|drt9|dnd }d }t!|drl|j"r`t!|j"dd rG|j"dj:}|j<|j>|j@|jBf}|jE| |d}t!|dr|jG|nd }tI|||||r|dnd ||||| }|jK||dz }|jLsy|jO|||fScc} } wcc} } w)z Parse document and extract all tables with enhanced metadata. Args: pdf_path: Path to PDF file Returns: Tuple of (document object, list of TableMetadata) FT)pipeline_options)format_optionsrprovpage_nor Ndatagridbbox)limitexport_to_markdown) table_index page_numberheadersconsolidated_headerdescriptive_texttable_markdownpreceding_textrowscolsfirst_cell_content)(r do_ocrdo_table_structuretable_structure_optionsdo_cell_matchingr rPDFr convertdocumentlist iterate_items enumerateidtableshasattrrrget_find_preceding_headersr_consolidate_headersr!num_rowsnum_colsr"lenstrr#ltrb_get_preceding_textr&r appendr_print_debug_info)rrr converterresultr7tables_metadatar'elements_with_depthitemdepth elements_listielemelement_to_indextablepage_num table_idxr)descriptive_textsr*r.r/ first_cell first_rowr#rHr-table_mdmetadatas rparse_documentz"EnhancedTableParser.parse_document1sH./"'.2+DH00A&'OO_N^-_` ""8,?? #8#9#9#;<1DE1D+$1D E7@7OP7OGAtBtHaK7OP__EE6*uzzgejjQRmU^>_ 1 %% ),,RY;IG " 1}-1-I-I-Yb-c**#' ((W&*&?&?&H#DDJuf%%**zz**zz**!8quzz62uzz:=ejjoo:NQR:REJJOOA$6XZ $Y!);>GlYq\):PTJDuf%%**APV9WJJqM&&QSS!##qss+"55mYVW5XN>EUL`=au//9gkH%'$$79J!21!5PT'-#- H  " "8 , 1 Kzz&&x0M%P((WFPs  M",M(elementsrXc g}g}d}d}tt|j|D]}||z dz }||} t| j} | dvr |j rt d| d||fS| dk(r|r|j r t d||fSt| d r | js| jj} | st| d xr| jdu} | r| j} |8| }|jd | |j st d | d | ddd| |k(r9|r7|jd | |j s.t d| d | dddE|j rt d| d|d||fS|sBt|d k(r4|jd | |j st d| ddd|sd}||fS)a Find headers that immediately precede a table. Only consolidates headers that are: 1. Consecutive (no non-header content between them) 2. Of the same level 3. Immediately before the table 4. Not separated by boundary elements (tables, images, etc.) Args: elements: List of document elements table_idx: Index of the table in the elements list Returns: Tuple of (headers list, descriptive texts list) NTr ) TableItem PictureItemz Hit boundary element (z), stopping header searchListItemz/ Hit list item after finding headers, stoppingtextlevelrz Found first header at level z: 2...z$ Found consecutive header at level z Found different level header (z vs z ), stoppingz Found descriptive text: F) rangeminrtype__name__rprintr=rdstripreinsertrC)rr_rXr)rY current_level consecutiverSidxrT elem_typerd is_headerres rr?z+EnhancedTableParser._find_preceding_headerss*"  s4,,i89Aa-!#CC=DT ++I88::4YK?XYZn)))iJ&7::KLb)))]4( 99??$D g.I4::T3II !($)MNN1d+zz >ugRSb {RUVWm+ NN1d+zz DUG2dSVTVi[X[\]zz @tM?Zefg )))3'8#9Q#>%,,Q5zz :49+SIJ"'K}:D)))rr)cd|syt|dk(r|dS|jj|S)z Consolidate multiple headers into a single string. Args: headers: List of header texts to consolidate Returns: Consolidated header string Nr r)rCrjoin)rr)s rr@z(EnhancedTableParser._consolidate_headerss5 w<1 1: ~~""7++rr%cg}d}tt|j|D]}||z dz }||}t|j} | dk(r|j dd=t |dsJ|jsW|jj} | st|j d| |dz }||k\sn|rdj|SdS)aY Get text content immediately preceding the table. Args: elements: List of document elements table_idx: Index of the table limit: Maximum number of text elements to include Returns: Preceding text context or None, with [Table] placeholders for intervening tables rr raz[Table]rd N) rhrirrjrkrnr=rdrmru) rr_rXr%textscountrSrqrTrrrds rrIz'EnhancedTableParser._get_preceding_textss4,,i89Aa-!#CC=DT ++IK' Q *tV$yy(LLD)QJE~!:$$)tyy2d2rr]ctd|jdzd|jd|jrtd|j|jrtd|j|j rtd|j dd d td |j d |jd |jrtd|jddd yy)z$Print debug information for a table.z === Table r z (Page z) ===zConsolidated Header: zIndividual Headers: zDescriptive Text: NdrgzSize: u rows × z colsz First Cell: rf) rlr'r(r*r)r+r.r/r0)rr]s rrKz%EnhancedTableParser._print_debug_info+s X11A56gh>R>R=SSXYZ  ' ' )(*F*F)GH I    ()9)9(:; <  $ $ &x'@'@#'F&GsK L x}}oYx}}oUCD  & & L! 3S 3rr)rrrtypingrrrrdocling.datamodel.base_modelsr"docling.datamodel.pipeline_optionsr docling.document_converterr r modelsr rrrrrs0 --5AI!D3D3r