=j>SrSSKJr SSKrSSKrSSKrSSKrSSKrSSKJ r SSK J r J r J r JrJr SSKJrJr SSKJr SSKJrJrJrJrJrJrJrJr SS KJr SS K J!r!J"r" \ "\#5RI5RJS r&\&S - r'S r(Sr)\ "\#5RI5RTS- r+\RX"S5r-"SS\5r."SS\5r/"SS\5r0"SS\5r1S(Sjr2Sr3S(Sjr4S(Sjr5S)Sjr6S*Sjr7S+Sjr8S,S jr9S-S!jr:S.S"jr;S/S#jr<S0S$jr=S1S%jr>\?S&:Xat\@"\R5S':a\ "\RS'5O\'rB\@"\R5S :a\C"\RS 5O\(rD\Rv"\;"\B\D55 gg)2a QUE-270 probe: can the Set-of-Mark vision model find footnote blocks? Question this answers --------------------- Today the per-table structure-correction step captures a footnote only when the definition block falls inside that table's crop, and `share_footnotes` then spreads it page-wide by marker string alone. That marker-only sharing mis-attributes when a page carries two independent footnote namespaces. This probe tests a grounded alternative: detect footnote BLOCKS visually as their own page regions, ground each to the PDF text layer, read its leading marker deterministically, and associate it to a table by GEOMETRY (the block sits directly below the table's column band), confirmed by the marker the table references. Two detector shapes are run on the SAME gridded page image so the visual model's capability can be compared directly: A. Separate footnote-only locator pass (one extra vision call; dedicated prompt). B. Extended locator that returns tables AND footnote blocks in one call. Nothing in the production package is modified. The probe reuses the prod scaffold read-only: `overlay_grid`, the grid geometry, `tighten_region`, `page_words`, and the real `PydanticAIGridLocator` for table regions in approach A. Run --- uv run python experiments/que270/footnote_finder_probe.py # Visa p11 uv run python experiments/que270/footnote_finder_probe.py Outputs a side-by-side report to stdout and writes the gridded page image plus a JSON dump of both runs under experiments/que270/out/. ) annotationsN)Path)AnyListOptionalSequenceTuple) BaseModelField) page_words) DEFAULT_COLS DEFAULT_ROWSPydanticAIGridLocator column_index column_labelsgrid_region_norm overlay_gridtighten_region) render_pages)DEFAULT_LLM_MODEL get_settingsz:.cache/s3/qubera-docs/visa/Q1-2026-Earnings-Release_vF.pdf outu/^\s*(?:\((\d{1,2})\)|(\d{1,2})\.|([*†‡§]))c\rSrSr%\"SSS9rS\S'\"\SS9rS \S '\"S S 9r S\S '\"SS 9r S\S'\"SS 9r S\S'\"SS 9r S\S'\"SSS9r S\S'Srg)FootnoteBlockFlagHz+1-based reading order of the footnote block)ge descriptionintordinalz@The marker each footnote line leads with, e.g. ['(1)','(2)','*'])default_factoryr!z List[str]markersz(Printed grid row at the block's TOP edge)r! row_startz+Printed grid row at the block's BOTTOM edgerow_endz3Printed grid column LETTER at the block's LEFT edgestr col_startz4Printed grid column LETTER at the block's RIGHT edgecol_endz&First few words of the block, verbatim)defaultr! text_previewN)__name__ __module__ __qualname____firstlineno__r r#__annotations__listr%r&r'r)r*r-__static_attributes__r.B/home/mande/repo/quber/experiments/que270/footnote_finder_probe.pyrrHsA+XYGSYVGY'QRIsR%RSGSS'\]Is]%[\GS\b6^_L#_r6rc,\rSrSr%\"\S9rS\S'Srg)FootnoteFinderResultUr$List[FootnoteBlockFlag]blocksr.N) r/r0r1r2r r4r=r3r5r.r6r7r9r9Us&+D&AF #Ar6r9cj\rSrSr%\"SS9rS\S'\"SS9rS\S 'S\S 'S\S 'S\S 'S\S 'Srg) TableFlagYr)r r"r#r+)r,r(titler&r'r)r*r.N) r/r0r1r2r r#r3rAr5r.r6r7r?r?Ys3A;GSr"E3"N LN Lr6r?cB\rSrSr%\"\S9rS\S'\"\S9rS\S'Sr g) CombinedResultbr;zList[TableFlag]tablesr<r=r.N) r/r0r1r2r r4rEr3r=r5r.r6r7rCrCbs!#D9FO9&+D&AF #Ar6rCc <[U5SnSUSUSUSUS3 $)Nz$The page image has a grid overlaid: z+ numbered horizontal rows (1 at the top to z2 at the bottom, labeled on BOTH side margins) and z$ lettered columns (A at the left to z at the right, labeled top and bottom). Read the printed row numbers and column letters off the grid; never estimate coordinates.)r)rowscolslasts r7_grid_preamblerKjsE  r "D .tf5I$P%%)F+[ \r6uA FOOTNOTE BLOCK is one or more short notes that explain a marker used elsewhere on the page. Each line leads with a marker: a parenthesized or superscript number ((1), (2), a raised 1/2), an asterisk *, or a dagger / double-dagger / section sign. Footnote blocks are visually distinct: smaller or lighter text, set off below a table by whitespace or a thin rule, usually near the bottom of the page. Do NOT report any of the following as a footnote block: - a bulleted list (lines led by a round bullet, dash, or square) — that is body text; - a numbered list that is the document's main content (full sentences, not tied to a marker that appears up in a table); - a table row, a column header, a title, a units/scale caption, or a prose paragraph; - page furniture (page number, running header, URL). The distinguishing test: a footnote's marker also appears attached to a header or cell somewhere in a table above it. A bullet's marker does not.c |[X5S[S[R"[R 5SS9S3$)Nz a Find every FOOTNOTE BLOCK on the page. For each, in reading order, report: - ordinal (1 = first, top-to-bottom) - markers: the marker each line in the block leads with, in order (e.g. ["(1)","(2)"]) - row_start / row_end: printed grid rows at the block's top and bottom edges - col_start / col_end: printed grid column letters at the block's left and right edges - text_preview: the first several words of the block, copied verbatim If the page has no footnote block, return an empty list. Never invent a footnote from a marker that has no distinct note text below the table. Return ONLY a JSON object conforming to this schema (no prose, no code fences): rindent )rK_FOOTNOTE_DEFINITIONjsondumpsr9model_json_schemarHrIs r7footnote_only_promptrUsJ 2 2 4Q?@A# r6c |[X5S[S[R"[R 5SS9S3$)Na~ Report TWO things on this page: its tables, and its footnote blocks. TABLES: a block where values line up in two or more vertical columns across rows. Bound the whole table (title, headers, row labels, body, totals). A bulleted or numbered list, a single column of figures, and prose are NOT tables. For each table report ordinal, title, row_start, row_end, col_start, col_end. a For each footnote block report ordinal, markers, row_start, row_end, col_start, col_end, text_preview (as in the footnote schema). Footnotes are NOT tables and must not appear in the tables list. Return ONLY a JSON object conforming to this schema (no prose, no code fences): rrMrO)rKrPrQrRrCrSrTs r7combined_promptrWsJ N , , .q9:;% r6c SSKJn SSKJn [ 5R nUR =(d [nUR(d [S5eSSK J n U"XTR5nU"UUUU"SS9S9$) Nr)Agent) ModelSettingsz>ANTHROPIC_AUTH_TOKEN not set; this probe needs the real model.)make_oauth_anthropic_modelg) temperature) output_type system_promptmodel_settings) pydantic_airYpydantic_ai.settingsrZrllmmodelranthropic_auth_token RuntimeErrorquber.agents._oauth_gater[)r^r]rYrZrbrcr[ anth_models r7 _build_agentrhsj!2 .  C II **E  # #[\\C+E3K3KLJ #$5  r6c# SSKJn [X5nU"UR5SS9nUR SU/5IShvN nUR $N7f)Nr) BinaryContentz image/png)data media_typezReport using the grid.)r`rjrh read_bytesrunoutput)r^r]griddedrjagentimageresults r7 _run_agentrtsN)  4E w113 LE996>? ?F ==@s=AAAcN[URU5n[URU5nXv:aXvpv[URUR XgXE5nUSSU- - U-n USSU- -U-n UV s/sH#oU SU S-S- s=::aU ::dMO M!U PM% n n /n [ U 5HnUV s/sHoSR5PM nn [U5S:dM6[RUS5(dMU[R"SUS5(dMvU R[S U555 M U (a[U 5S - OUSU-nU nUV s/sH$n UU SU S-S- s=::aU::dMO M"U PM& nn USUU- USUU- 4nURS S 9 S RSU55n[RU5nU(a![!SUR#55S5OSnUUU4$s sn fs sn fs sn f)zPCoarse grid flag -> tight box, grounded text, and leading marker from PDF words.rg@g?rrz ^[A-Za-z]c3*# UH oSv M g7f)rNr..0ws r7 _ground_block..s$84aqT4g?c@[USUS-S- S- 5US4$)Nrrvrrwr)roundr{s r7_ground_block..s(uadQqTkQ%6%:;QqTBr6key c3*# UH oSv M g7f)rwNr.rys r7r|r}s)&QaD&r~c36# UHo(dM Uv M g7fNr.)rzgs r7r|r}s.jA11js  N)rr)r*rr&r' _group_linesstriplen_LEADING_MARKERmatchreappendminsortjoinnextgroups)flagwordspage_wpage_hrHrIc0c1coarse search_top search_botr{band def_line_topslinetokstop_ybot_yinsideboxtextmmarkers r7 _ground_blockrs  dnnd +B dllD )B wB dnndllBD OF)cDj(F2J)cDj(F2J Nu!qtad{a.? M: MA MAuD N!#MT"&*+d! d+ t9>o33DG<<,X\]^X_A`A`  $84$8!8 9#+8S # %VAY=OE E FA%AaD1Q4K1+<"E"Ea"EaF F !9efnfQi @C KKBKC 88)&) )Dd#A9:T.ahhj. 5F f  O, Gs*6HHH6H-H"H"H"c,/n[USS9HinUSUS-S- nU(a?[X1SSSUSSS-S- - 5S::aUSRU5 MWURU/5 Mk UHnURS S9 M U$) zLCluster words into text lines by vertical center, each sorted left-to-right.cUSUS-S- $)Nrrvrr.rs r7r_group_lines..s!A$1+):r6rrrvrrGrg@c US$)Nrr.rs r7rr s!r6)sortedabsrr)rlinesr{ycrs r7rrsACE E: ;dQqTkQ  SRy|Ar1a@AEEF#M "I  Q  LL!  <  n % Lr6cj[USUS5[USUS5[USUS5[USUS54up#pESnSnUH_unupp[X5[X5[X5pn [XN5[X-5- nUS::aMHX?- nUS:dMTUU:dM\UUpgMa U$)zQOrdinal of the nearest table whose column band overlaps and sits ABOVE the block.rrrrvNgeAg{Gz)rmax) block_boxrEbx0by0bx1_bestbest_gapr#tx0ty0tx1ty1t_x0t_x1t_bottom x_overlapgaps r7 _associaters IaL)A,7Yq\9UV<9XIaL)A,7Yq\9UV<9XZCcDH)/%%#C"3}c#mS]HNS^3 > n %APPROACH B — extended locator (tables + footnotes, one call)zcombined call reported z table(s), z footnote block(s)z -runs.json)r#region) approach_a approach_b real_tablesrrMz Wrote zGridded image: )$OUT_DIRmkdirprintrrbrcrDPIr tempfileTemporaryDirectoryrrstemrrr rlocater#rr_fmtrtrUr9_report_blocksr=rWrCrE _flag_region write_textrQrR model_dump)pdfpagerrrtmpimagespage_imgrplocatorlocatedtrr#rabb_tablesdumpors r7rnrn&s MM$M. F3%. F4& <>#5#5#;#;#P?P"QQYZ]Y^ _` (O&s1FE  $ $I 6#c3S 2(#sxxj4& ==X lC()xd;;6=>g 188,g > *3{+;* + o BC h1, MOcelm mqxxD o NO h_\<H.Zab bVWV^V^_V^QRYY Q l KLV^_ 'AHH k#ahh-PbcdqxxA; 7> zD64 4DOO llnllnHST 1A ; T      HTF  OG9 %&G<> n c_7 7 6J UsBM)A/ML?M !M,BM2M3A"MM M(*M AM$AM)M#*M)?MM M M M  M)c[URU5[URU5pCXC:aXCpC[URUR X4X5$r)rr)r*rr&r')rrHrIrrs r7rr\sB $..$ /dllD1Q wB DNNDLL"$ MMr6c U(d [S5 gUHn[XQX#[[5upgnU(a [ Xd5OSn [ U5S:aUSSS-OUn [SUR SUR35 [S[U535 [SU<S35 [S U S 35 [S U 35 M g) Nz (no footnote blocks reported)dz...z block z: model markers=z grounded region : z grounded marker : z% (read from PDF text, not the model)z associated table: z+ (nearest table above with column overlap)z grounded text : ) rrrr rrr#r%r) r=rrrrEblkrrrownerpreviews r7rrcs  /0)#flT`a6+. 3'D*-d)c/4:%t  %5ckk]CD &tCyk23 &vj0UVW &ug-XYZ &wi01r6cFUcgSSRSU55-S-$)NNone(z, c3(# UHoSv M g7f)z.3fNr.)rzvs r7r|_fmt..{s6v!W:vs))r)rs r7rrxs) ~ 6v66 6 <rs3@# 77%0   ?: H~''*QQ    x. " ) )E 1**OP ` `B9B BYB A",4&( ( ;( (  (  (  (L(V  ; 7  0 C.3'lN2 '2 ;2 2  2 D 2  2*=  z#&sxx=1#4d388A;+G#&sxx=1#4s388A;,H KKGX&'r6