1j
1 S r SSKJr SSKrSSKJr SSKJrJrJ r J
r
J r SSK J
r
JrJrJrJrJr SSKJrJrJrJrJrJr " S S \5 r\R8 " S
5 r \R8 " S \R< 5 r \R8 " S \R< 5 r SS
jr! S SS jjr" S S S jjr#S S jr$S S jr%S S jr&S!S jr'S"S jr( " S S\5 r)S#S jr*S$S jr+g)%u Turn one page's dpt-3 response into records, and project them for the graft.
A dpt-3 response has two parts: one long markdown string holding all the text
the model read off the page, and a tree describing what sits where. Each tree
node names its kind, gives its rectangle on the page, and points at the stretch
of the long string that belongs to it — the node itself stores no text, so
getting a node's text means cutting its stretch out of the string.
`digest_page` walks the tree and produces the track's records: a figure node
becomes a `DigestedFigure` with its kind, description and value tables kept
separate; a table node and its cell nodes become a `DigestedTable` with a box
on every cell; everything else on the page is kept as context. A response
without the tree raises `UnrecognizedResponse` naming the stored file and the
model, so a billed page can never pass as blank because the code did not
recognize the format. A page node reporting anything but success raises too,
with the response's own reason.
`project_scan` turns a digest into the `PageScan` every downstream consumer
already reads — the graft, the correction sweep, the value reconciliation. A
figure is projected as its description followed by its value tables rendered
as markdown; a table is projected grid-for-grid. The rest of the digest stays
in the track models.
The figure markup is the response's own labelling — the model wraps each
figure's content in tags it emits itself — so reading it back is reading the
format, not inferring anything about the page.
) annotationsN)
HTMLParser)AnyDictListOptionalTuple)DigestedFigure
DigestedTable DigestedTextFigureValueCellFigureValueTable
PageDigest)Box ChartContext FigureRecordPageScan
PageStatus ScannedTablec \ rS rSrSrSrg)UnrecognizedResponse. z5A response in neither the dpt-2 nor the dpt-3 format. N)__name__
__module__ __qualname____firstlineno____doc____static_attributes__r ]*\btype="([^"]*)"[^>]*>z (.*?)zc [ U R S5 [ 5 ( a g[ U R S5 [ 5 ( a gg)zWhich format a raw response holds: `dpt2`, `dpt3`, or None for neither.
The format is visible in the response itself: a dpt-2 response carries a
flat `chunks` list, a dpt-3 response carries the `structure` tree.
chunksdpt2 structuredpt3N)
isinstancegetlistdict)responses r! response_generationr, < s9 (,,x($//(,,{+T22 r c
U R S5 n[ U[ 5 ( d [ U=( d S SU S35 eU R S5 =( d SnU R S5 =( d 0 n[ UUR S5 UUR S 5 =( d UR S
5 UR S 5 =( d 0 R S UR S
5 5 S9nUR S5 =( d / GH> nUR S5 S:w a M UR S5 n U S; a0 [ U=( d S SU SU < SUR S5 S35 eUR S5 =( d / H n
U
R S5 n U S:X a' UR R [ XU5 5 MA U S:X a' UR R [ XU5 5 Mn UR R [ UU =( d S[ X5 [ U
5 U
R S5 S95 M GMA U$ ) a/ Digest one page's raw dpt-3 response into the track's records.
`page` is the 1-based source page the submission came from; the response
numbers its own single page from one, so the source page is carried in
rather than read back out of it. `artifact` names the stored response file
for error messages.
r% z the responsez (model z>) holds neither a dpt-2 chunks list nor a dpt-3 structure treemarkdown metadatajob_id
model_versionversionbilling
total_credits credit_usage)pager1 modelr3 creditschildrentyper7 status)Nokz) reports page status z (reasonz/); the page was billed and cannot pass as blankfiguretableid)r7 kindtextboxnode_id)r( r' r* r r
ValueErrorfiguresappend_figuretables_tablecontextr _slice_box) r+ r7 r8 artifactr% r. metadigest page_noder< noderB s r! digest_pagerT I s [)I i & &")>*(5' :)
)
||J'-2H <<
# )rD
xx! )@TXXi-@)$*//.AYZ F ]]:.4"4 == F * x( % -~.hug=STZS]Y]]8,--\^
MM*-33D88F#Dx %%gdd&CD
$$VDD%AB%% ! !ZR #D 3 J $ 4 52 Mr c @ U R Vs/ s HA n[ U R [ U5 UR UR
U R S9PMC nnU R Vs/ s HM n[ U R UR UR UR UR
U R S9PMO nnU R Vs/ s Hc nUR R 5 ( d M$ [ U R UR UR UR UR
S9PMe n nU( a SO
U( a SOSn
[! U R U
UU R U R" U R$ U R& UUUUU S9 $ s snf s snf s snf )u The digest as the `PageScan` every downstream consumer already reads.
The figure records carry text, id and rectangle — the graft's whole
contract. The digest keeps everything else.
)r7 rC rD chunk_idr1 )r7 cells
cell_boxesrD rV r1 )r7 rB rC rD rV rG rJ empty) r7 r< picture_classesr1 r8 r3 r9 reusedresponse_artifactrG rJ rL )rG r r7 figure_textrD rE r1 rJ r rW rX rL rC stripr rB r r8 r3 r9 ) rQ rZ r\ r[ r? rG r@ rJ itemrL r< s r! project_scanr` sn & nn %F V$
^^==
% & ]]
#E ++ '' ]]==
#
NN"D 99?? l &++DIIDII488^b^j^jk"
'.v87F
[['}}ll+
7
s AF'AF #F2AFc ( / nU R R 5 ( a) UR U R R 5 5 U R H( n[ U5 nU( d M UR U5 M* SR U5 $ )a A figure's readable content: its description, then its value tables.
This is what the graft puts on the picture and what the correction agent
and the value reconciliation read, so the values are rendered as markdown
rows rather than left in the response's markup.
) descriptionr^ rH values_markdown_rowsjoin)r? partsr@ rendereds r! r] r] ss E
!!
V''--/0 !%( 8 LL " ;;u r c / nU R H nU Vs/ s H< nSR UR R 5 5 R SS5 PM> nnUR SSR U5 - S- 5 [
U5 S:X d M [
U R 5 S: d M UR SS[
U5 - - 5 M S R U5 $ s snf )
z7A value table as pipe-delimited rows, one line per row. |/z| z | z | z --- |
)rowsrf rC splitreplacerH len)r@ linesrowcellrW s r! re re s EzzKNO34$))//+,44S#>3O
TEJJu --45 u:?s5:: 2 LLx#c( 22 3
99U Ps AC"c
J [ X5 n[ R U5 n[ R U5 Vs/ s H) oUR 5 ( d M UR 5 PM+ nn[ R U5 Vs/ s H n[ U5 PM nn[ UU( a UR S5 R 5 OSSR U5 [ U 5 U V s/ s H oR ( d M U PM sn U R S5 S9$ s snf s snf s sn f )zHOne figure node: kind, descriptions and value tables out of its stretch.rm r/ rb rA )r7 rB rc rD rd rE )rM _FIGURE_OPEN_REsearch_DESCRIPTION_REfindallr^ _VALUE_TABLE_RE _value_tabler
grouplowerrf rN ro r( )
rS r. r7 content
kind_matchd descriptionshtmlrJ ts
r! rI rI s T$G ''0J'6'>'>w'GU'G!779IAGGI'GLU-<-D-DW-M
N-MTl4 -MF
N
-7j q!'')RKK - J!,6aVV6,
V
N -s DDD#D ;D c , / nSnSnU R S5 =( d / GH nUR S5 S:w a M [ UR S5 =( d S5 n[ UR S5 =( d S5 n[ XG[ UR S5 =( d S5 - 5 n[ XX[ UR S 5 =( d S5 - 5 nUR XxS
R [ Xa5 R
5 5 [ U5 45 GM [ U5 V s/ s H n [ U5 V s/ s H n S PM sn PM! n
n [ U5 V s/ s H n [ U5 V s/ s H n S PM sn PM! n n U H: u pxpSUs=:: a U: d M O M SUs=:: a U: d M( O M, XU U' XU U' M< U( do [ [ X5 5 nUR VVs/ s H ow Vs/ s H ofR PM snPM n
nnU
VV s/ s H ow V s/ s H n S PM sn PM n nn [ X*U [ U 5 U R S
5 S9$ s sn f s sn f s sn f s sn f s snf s snnf s sn f s sn nf )zOne table node as a dense grid, cells and boxes read off its cell nodes.
A cell that spans is written at its own row and column; the positions it
covers stay blank and carry no box, so the grid never states a value twice.
r r: r;
table_cellrt colrowspanrm colspanrj r/ NrA )r7 rW rX rD rE )
r( intmaxrH rf rM rp rN ranger| ro rC r )rS r. r7 placedro columnsru rt r _gridboxesrC rD value_tables r! rK rK s" 9;F DG$** 88F | + $((5/&Q'$((5/&Q'4s488I#6#;! <<=gS))<)A%B BC
s&*@*F*F*H!I4PT:VW + 38+ >+Qw ( AR (+D >PUVZP['\P[1uW~(F~!~(FP[E'\ %$ ?d??qC 1' 1 1 !IcN !#JsO &
"6$#9: 7B7G7GH7Gs+sts+7GH045$1$$5 d5d4jZ^ZbZbcgZh ii
) >(F'\ ,H$5s` 5I1 I,
I1*I;= I6
I;< JJ J) J2 J >J,I16I; J Jc
[ 5 nUR U 5 UR 5 [ UR VVs/ s H% o"( d M U Vs/ s H n[ US9PM snPM' snnS9$ s snf s snnf )zDOne value table's rows, read out of the response's own table markup.)rC )ro ) _GridTextfeedcloser ro r
)r parserrt ru s r! r| r| sb
[F
KK
LLN Z`ZeZe!mZeSVil"N#"N#$?#=#"NZe!m nn"N!ms
A7
A7
A2%A7
2A7
c r ^ \ rS rSrSrS U 4S jjrS S jrS
S jrSS jrS U 4S jjr S S jr
S S jr S
r U =r
$ )r i zCollects `td`/`th` text row by row. Markup inside a cell is dropped and
its text kept, so a value typeset with a line break reads as one string.c J > [ TU ] SS9 / U l S U l S U l g )NT)convert_charrefs)super__init__ro _row_partsself __class__s r! r _GridText.__init__ s(
$/%' )- +/ r c US:X a U R 5 / U l g US; a U R 5 / U l g US:X a* U R b U R R S5 g g g )Ntrtdthbrrj )
_flush_rowr _flush_cellr rH )r tagattrss r! handle_starttag_GridText.handle_starttag s^ $; OO DI
L
DK
D[T[[ 4 KK s # 5[r c ` US; a U R 5 g US:X a U R 5 g g )Nr r )r r )r r s r!
handle_endtag_GridText.handle_endtag" s, ,
D[ OO r c V U R b U R R U5 g g N)r rH )r datas r! handle_data_GridText.handle_data( s# ;; " KK t $ #r c B > [ TU ] 5 U R 5 g r )r r r r s r! r _GridText.close, s
r c U R c g U R c / U l U R R SR SR U R 5 R 5 5 5 S U l g )Nrj r/ )r r rH rf rp r s r! r _GridText._flush_cell0 sX ;; 99 DI "''$++"6"<"<">?@ r c U R 5 U R b- U R R U R 5 S U l g g r )r r ro rH r s r! r _GridText._flush_row8 s: 99 II TYY ' DI !r )r r ro )returnNone)r strr z List[Tuple[str, Optional[str]]]r r )r r r r )r r r r )r r r r r r r r r r r r r
__classcell__)r s @r! r r s1 P0 $ % r r c U R S5 =( d 0 nUR S5 =( d 0 nUR S5 UR S5 pTUb Uc gXU $ )zJA node's text: the stretch of the response's one long string it points at. groundingr startendr/ )r( )rS r. r rngr r s r! rM rM ? sW %+I
--
&BC!3775>3 } # r c U R S5 =( d 0 nUR S5 =( d 0 n US US US US S.$ ! [ a gf = f) zGA node's rectangle in the workflow's own box keys, or None without one.r rD xminyminxmaxymax)lefttoprightbottomN)r( KeyError)rS r rD s r! rN rN I sg %+I
--
$"CKv;[&k
s A
AA)r+ Dict[str, Any]r
Optional[str]r )
r+ r r7 r r8 r rO r r r )NF)
rQ r rZ z List[str]r\ r r[ boolr r )r? r
r r )r@ r r r )rS r r. r r7 r r r
)rS r r. r r7 r r r )r r r r )rS r r. r r r )rS r r z
Optional[Box]),r
__future__r re html.parserr typingr r r r r quber.core.figures.dpt3.modelsr
r r r
r r quber.core.figures.modelsr r r r r r rF r compilerw DOTALLry r{ r, rT r` r] re rI rK r| r rM rN r r r! r s( 8 # " 3 3 j i@: @
** DE ** @"))L ** 4bii@
" # 6 6
6 6 6
6x (, 3 3 3 %3
3
3l "
jDo.
. b r