j S r SSKrSSKrSSKrSSKrSSKrSSKJr SSKJ r SSK
J r SSK r SSK
Jr SSKJrJr SSKJr SS KJr SS
KJrJr SS KJr \ R6 " 5 \ R8 " \" S 5 S9S
5 5 r \ R= 5 \ R> " S5 \ R@ " SSSSS9\ R@ " SS\ RB " \ V s/ s H o RD PM sn 5 SS9\ R@ " SSSS9\ R@ " S S \#S S 9\ R@ " S \ RB " S!S"/5 S!S#S$9\ R@ " S%S&S'S(S)9S\$S*\$S+\ \$ S,\ \$ S-\#S.\$S/\%4S0 j5 5 5 5 5 5 5 5 r&\ R= 5 \ R> " S1\ R " S'S2S'S39S49\ R@ " SSSS5S9\ R@ " SS\ RB " \ V s/ s H o RD PM sn 5 S6S9\ R@ " SSS7S9\ R@ " S S \#S8S 9\ R@ " S9S:S;S9S1\$S*\$S+\ \$ S,\ \$ S-\#S<\$4 S= j5 5 5 5 5 5 5 r'\ R= 5 \ R@ " SS\ RB " \ V s/ s H o RD PM sn 5 S>S9\ R@ " S?S@S'SAS)9S+\ \$ SB\%4SC j5 5 5 r(\ R7 5 SD 5 r)\)R= 5 \ R@ " SES'SFS)9\ R@ " S%S&S'S(S)9SG\%S/\%4SH j5 5 5 r*\)R= 5 \ R> " SI\ R " S'SJ9S49\ R@ " SKS'SLS)9\ R@ " S%S&S'S(S)9SI\$SM\%S/\%4SN j5 5 5 5 r+\)R= 5 \ R> " S1\ R " S'S2S'S39S49\ R@ " S9SOSPS9\ R@ " SKS'SLS)9\ R@ " S%S&S'S(S)9S1\$S<\$SM\%S/\%4SQ j5 5 5 5 5 r,\)R= 5 \ R@ " S%S&S'S(S)9S/\%4SR j5 5 r-\)R= 5 \ R@ " S\ RB " SSST/5 SUS9\ R@ " SV\ RB " SWSX/5 SYS9\ R@ " SZS[\#S\S 9\ R@ " S%S&S'S(S)9S+\ \$ S]\ \$ S^\#S/\%4S_ j5 5 5 5 5 r.\ R= S`Sa9\ R> " S5 \ R@ " SSSSbS9\ R@ " ScSd\ RB " / SeQ5 SfSgS$9\ R@ " ShS'SiS)9\ R@ " Sj\ RB " / SkQ5 SlSmS$9\ R@ " S \ RB " S!S"/5 S!S#S$9\ R@ " SV\ RB " / SnQ5 SoSpS$9S\$S*\$Sd\$Sq\%Sr\$S.\$S]\$4Ss j5 5 5 5 5 5 5 5 r/\ R= StSa9\ R> " Su5 \ R@ " SSSSbS9\ R@ " SSSvS9\ R@ " Sw\#SSxS$9Su\$S*\$S,\ \$ Sy\ \# 4Sz j5 5 5 5 5 r0\ R= S{Sa9\ R> " Su5 \ R@ " S|S'S}S~9\ R@ " SSSSbS9\ R@ " SSSS9\ R@ " S\ RB " SSB/5 SSS$9Su\$S\$S*\$S,\ \$ S\$4
S j5 5 5 5 5 5 r1\ R= SSa9\ R> " S5 \ R@ " SSSSbS9\ R@ " S\ RB " / SQ5 SSS$9\ R@ " SS'SS)9\ R@ " SS\#SS 9\ R@ " S\ RB " / SQ5 SSS$9\ R@ " SS'SS)9S\$S*\$S\ \$ S\%S\#S\$S\%4S j5 5 5 5 5 5 5 5 r2\ R= 5 \ R> " S5 \ R@ " SSSSS9\ R@ " SS\#SS 9\ R@ " SS'SS)9S\$S*\$S\#S\%4S j5 5 5 5 5 r3\ R= 5 \ R> " S5 \ R@ " SSSSbS9\ R@ " SSSS9\ R@ " S\ RB " / SQ5 SSS$9\ R@ " SS\#SS 9\ R@ " S \ RB " S!S"/5 S!S9\ R@ " Sj\ RB " / SkQ5 SSS$9\ R@ " SS'SS)9S\$S*\$S\ \$ S\ \$ S\#S.\$Sr\$S\%4S j5 5 5 5 5 5 5 5 5 r4S\ \$ S\$Su\ S\S\$S.\$S\#Sr\$S\%S*\$4S jr5S.\$4S jr6\ R= 5 \ R> " S\ R " S'SJ9S49\ R@ " S\ RB " / SQ5 SSS$9\ R@ " SS\#SS 9S\$S\ \$ S\#4S j5 5 5 5 r7\ R= 5 \ R@ " SSSS9\ R@ " SS\#SS 9\ R@ " SS'SS)9S\$S\#S\%4S j5 5 5 5 r8\ Rs \/SSa9 \ Rs \2SSa9 S r:\;S:X a \:" 5 ggs sn f s sn f s sn f )u
Command-line interface for Quber.
Four commands compose into the extraction pipeline: `document`, `table`, `fuse`,
`figure`. They hand work to each other through files rather than in memory, and
each names its output after the source document's stem, so one stage's
`--output-dir` is the next stage's input directory.
`table`, `fuse` and `figure` each finish by writing a completion marker,
`.tables.complete.json`, `.fuse.complete.json` or
`.figure.complete.json`, as their last upload. The marker carries status
`complete` or `failed`, so a downstream step learns from its existence that the
job's artifacts are whole or that the job failed. `document` writes no marker.
On the cloud path the parse runs on the RunPod worker (`runpod/handler.py`),
which writes the parse's marker itself.
`document` parses the PDF and writes `.docling.json` with
`.confidence.json` and `.cells.json` beside it. All three are always
written, because the later stages reload them together through
`ParseResult.load`. Markdown and split-page HTML are optional extras.
`table` extracts the tables and writes `.tables.json`, plus
`.flags.json` when any table carries a review flag: a corrected cell with
an inspect status, header text left out of the table, or a footnote marker that
could not be tied to a cell (`quber.core.extractors.base.cell_flags`).
`fuse` merges the two into `.unified.json` and `.unified.md`, with
`.cells.json` beside them — the parse's positioned page cells, which the
figure stage's value reconciliation requires next to the parse it is handed. It
also writes the fused tables to `.tables.json`, replacing any already in
the output directory, the fusion report `.fusion.json`, and
`.flags.json` when the tables or the heading review raised a flag. By
default it runs both engines in-process. Given `--artifacts-dir` it reloads
`.{docling,confidence,cells}.json` and `.tables.json` from a prior
run and only fuses, which is how the cloud path splits the work across jobs.
`figure` runs last, after fusion, so its enrichment lands in the document
everything downstream reads. It loads the parse back — `.unified.json`
first, then `.docling.json` — reads the pages holding charts, pictures,
and image-printed tables, and writes the enriched parse back under the filename
it read. Alongside it go `.figures.json` and one `.p.ade.json`
per page scanned, so a re-run reuses a page already paid for instead of
submitting it again. Three more are written only when they have content:
`.scanned-tables.json` when a table was read off a page image,
`.figure-values.json` when `.cells.json` sat beside the parse and
the value step is on — each plotted value reconciled between the scan's reading
and a local read grounded in those cells, with a status and a page box per
value — and `.figure-digest.json` when the scan model is dpt-3, the
default.
`ade`, `analyze`, `batch`, `dual`, and `validate` are standalone and not part of
that chain. `playground` serves the answering web app in `quber.playground`,
which runs the pipeline as subprocesses when a document is uploaded. With no
GPU worker configured it runs `fuse`, then `figure`, then its own ingest. With
one configured it runs `table` locally, sends the parse to the RunPod worker,
then runs `fuse --artifacts-dir`, `figure` and the ingest.
Nversion)Path)Optional)logger) ModelConfig
ModelProvider) LLMClient)resolve_document) output_sinkwrite_completion_marker) get_settingsquberc g)z7Quber - Advanced table extraction and inference system.N r '/home/mande/repo/quber/src/quber/cli.pyclir S s r pdf_pathz --output-dirz-ooutputz"Output directory (default: output))defaulthelpz
--providerz-pz6Model provider to use (auto-detected if not specified))typer z--modelz-mz>Specific model to use (uses provider default if not specified))r z--max-concurrent z.Maximum concurrent table analyses (default: 5))r r r z--presetztuned-financiallegacyzUDoclingParser configuration preset (default: tuned-financial; 'legacy' = pre-QUE-218))r r r z --verbosez-vTzEnable verbose logging)is_flagr
output_dirprovidermodelmax_concurrentpresetverbosec ^^^ U( a SOSn[ R " 5 [ R " [ R US9 [ 5 R
XtS.S9nU( a [ U5 OSn SSKJ n
U
" U UUUS 9m[ U 5 m[ R " SU 35 UUU4S jn [ R " U " 5 5 u pn[ R " S5 [ R " SU
S S35 [ R " SU 35 [ R " SU 35 g! [ a8 n [ R " S
U 3S S 9 [ R " S
5 Sn A NSn A ff = f)z
Analyze tables in a PDF document with LLM inference.
Example:
quber analyze document.pdf
quber analyze document.pdf --provider anthropic --model claude-3-opus-20240229
DEBUGINFOlevel) log_levelmax_concurrent_tablesupdateNr TableInferenceProcessor)r r settingsr! Error: Terr z
Analyzing: c z ># TR T5 I S h vN n [ T5 nUR SSS9 TR nX S3- nX S3- nTR U [ U5 5 TR
U [ U5 5 XU4$ Ns! [ a n[ R " SU 35 e S nAff = f7f)NTparentsexist_ok_analysis.json _summary.mdzAnalysis failed: )
process_documentr mkdirstem save_jsonstr save_summary Exceptionr error) results output_path base_name json_file summary_fileedocr processors r run_analysis analyze..run_analysis s % 6 6s ;;G z*K dT : I# >&BBI&;k)BBL
Y 8 " "7C ,= >|3 3 < LL,QC0 1 s9 B;B B
A2B B;
B
B8B33B88B;u
✓ Analysis completez
- Found total_tables tablesz - JSON output: z
- Summary: )r removeaddsysstderrr
model_copyr quber.processorsr-
ValueErrorclickechoexitr asynciorun)r r r r r r! r" r( r.
provider_enumr- rF rI rA rD rE rG rH s ` @@r analyzerZ Z s; F #I
MMO
JJszz+ ~(( &P ) H
08M(+TM 9 +"
8
$C
JJ xj)*, (/{{<>'B$G
JJ() JJGN 34G<= JJ"9+./ JJ |n-.O
WQC=d+ s ; D4 4
E6>.E11E6 directoryF)exists file_okaydir_okay)r z.Output directory for results (default: output)zModel provider to usezSpecific model to usez;Maximum concurrent table analyses per document (default: 5)z --patternz*.pdfz&File pattern to match (default: *.pdf)patternc ^^^ [ 5 R SU0S9n[ U 5 n[ UR U5 5 mT( d. [
R " SU 3SS9 [ R " S5 [
R " S[ T5 S35 [ U5 mTR SSS 9 U( a [ U5 OS
nS S K J
n U " XUS
9mUUU4S jn [ R " U " 5 5 n [ U 5 n
[# S U 5 5 n[
R " S5 [
R " SU
S[ T5 S35 [
R " SU S35 [
R " SU S35 g
! [ a8 n
[
R " SU
3SS9 [ R " S5 S
n
A
NS
n
A
ff = f)z
Process multiple PDF documents in a directory.
Example:
quber batch documents/
quber batch documents/ --pattern "*.pdf" --output-dir results/
r) r* z!No files found matching pattern: Tr0 r2 Found z PDF files to processr4 Nr r, )r r r. r/ c P ># / n T H n[ R " SUR 35 T R [ U5 5 I S h vN nUR
nTU S3- nTU S3- nT R
U[ U5 5 T R U[ U5 5 [ R " SUS S35 U R U5 M U $ N! [ a@ n[ R " SU 3SS 9 [ R " S
U S U 35 S nAGM S nAff = f7f) Nz
Processing: r7 r8 u ✓ rK z tables analyzedu ✗ Failed: Tr0 zFailed to process : ) rT rU namer9 r= r; r< r> appendr? r r@ )
rA pdf_fileresultrC json_path summary_pathrF rB pdf_filesrH s
r
process_batch batch..process_batch s !H JJ
7 8
C ( 9 9#h- HH %MM 'YK~*F F * {+-F F ##FC N;&&vs$:#;;KLMv& "( # I
C
^A3/T: 1(2aS ABB
CsA +D& C C
BCD&C
D##4D D& D##D&c 3 * # U H oS v M g7f)rK Nr ).0rs r batch.. s :'Q ('s u
✓ Batch processing complete!z - Processed /z
documentsz
- Analyzed
total tablesz - Results saved to: )r rQ r listglobrT rU rO rV lenr: r rR r- rS rW rX sum)r[ r r r r r_ r. dir_pathrY r- rF rk rA total_documentsrK rB rj rH s @@@r batchrz s < ~((1H.0Y(ZH IHX]]7+,I
6wi@dK JJI''<=> z"KdT2 08M(+TM 9+]Zbc 0 kk-/*G 'lO:'::L JJ12 JJ 0#i.1ALM JJ |nM:; JJ'
|156M
WQC=d+ s F
G
.GG
z&Show models for specific provider onlyz--allz-az"Show all models (can be very long)allc SSK Jn U" SS9nUR 5 nU ( a
[ U 5 /nO[ [ 5 n[
R " S5 [
R " S5 [
R " S5 [
R " S5 U GH= nUR U5 nUR U; nU( a S OS
n [
R " S UR R 5 S U 35 [
R " S
5 XvR n
U( d# [ U
5 S: a U
SS n [ U
5 S-
n OU
n Sn U H_ n
U
[ R R U5 :X a SOSn[ U
5 S: a U
SS S- nOU
n[
R " SU U 35 Ma U S: d GM# [
R " SU S35 GM@ U( ds [
R " S5 [ HR n[ R R U5 nU( d M+ [
R " SU SUR S35 MT gg)z(List available models for each provider.r ) AgentFactoryF)enable_logfirez%Model Providers and Available Models:2==================================================z?
Note: You can use any model name directly, even if not listed.zEThe list below shows models recognized by the installed pydantic-ai.
u ✓ (API key found)u ✗ (no API key)
z(----------------------------------------
Nz
(default) < 9 z... • z
... and z# more models (use --all to see all)u9
⚠ No API keys found. Please set environment variables:z (for )) quber.agentsr} get_available_providersr rt rT rU list_available_modelsvalueupperrv r DEFAULT_MODELSget API_KEY_VARS)r r{ r} factory availableproviders_to_showpmodels is_availablestatus
model_listdisplay_models remainingr r
model_displaykey_vars r r r s *%0G //1I *8 4 5 / JJ67 JJx JJQR JJWX
..q1ww)+ *6& r c g)z Database management commands.Nr r r r dbr ^ s r z--dropz9Drop existing tables before creating (WARNING: data loss)dropc SSK JnJn U( a SOSn[ R " 5 [ R
" [ R US9 [ R " S5 U" 5 nU" XPS9 [ R " S5 g
! [ a8 n[ R " S U 3S
S 9 [ R " S 5 S
nAg
S
nAff = f)z Initialize the database schema.r )
get_engineinit_dbr$ r% r& zInitializing database...)drop_allu% ✓ Database initialized successfullyu# ✗ Failed to initialize database: Tr0 r2 N) quber.db.connectionr r r rM rN rO rP rT rU r? rV )r r" r r r( enginerF s r initr d s
8"I
MMO
JJszz+ JJ)* &
:;
8<$G s ##B
C .CC rh )r\ z--generate-embeddingsz!Generate embeddings during importgenerate_embeddingsc SSK Jn U( a SOSn[ R " 5 [ R " [
R US9 U" XS9u pVU( a SOSn[ R " S UR S
U S U 35 g! [ a8 n[ R " S U 3S
S9 [
R " S5 SnAgSnAff = f)z5Import a single JSON analysis file into the database.r )import_json_filer$ r% r& )r with embeddingsr
✓ Imported rc rL ✗ Import failed: Tr0 r2 N) quber.db.importerr r rM rN rO rP rT rU filenamer? rV ) rh r r" r r( rG table_countemb_msgrF s r import_jsonr y s 3"I
MMO
JJszz++I_(;$
]3<<.;-wwiPQ
(,$7 #?@A
%j1E&F%GHI k " JJ~ &!+.
VC
O#4Bs8}oYsSZ|n\bcd / #
3A37TB s C D) )
E+3.E&&E+localopenaiz&Embedding provider (default: from env)z--devicecudacpuz&Device for local model (default: cuda)z --batch-size z'Batch size for processing (default: 32)device
batch_sizec SSK JnJnJn U( a SOSn[ R
" 5 [ R " [ R US9 [ R " S5 U" XS9n[ R " S5 U" 5 n U R U5 R 5 n
[ U
5 n U S:X a [ R " S 5 S
S
S
5 g
[ R " S U S 35 Sn [ U
S
5 H u pUR ( d M UR ( d M+ UR# UR UR 5 nUR% 5 Ul UR% 5 Ul U S
-
n X- S:X d M U R+ 5 [ R " SU
SU S35 M U R+ 5 [ R " SU SU S35 S
S
S
5 g
! , ( d f g
= f! [, aN n[ R " SU 3SS9 [ R. " U5 [ R0 " S
5 S
nAg
S
nAff = f)z3Generate embeddings for all tables in the database.r )ExtractedTableget_embedding_service get_sessionr$ r% r& z!Initializing embedding service...)r r z Loading tables from database...zNo tables found in databaseNra z tables to processr2 z Processed rr z
tables...u ✓ Generated embeddings for rL u# ✗ Failed to generate embeddings: Tr0 )quber.dbr r r r rM rN rO rP rT rU queryr{ rv enumerate llm_titlellm_descriptionembed_table_metadatatolisttitle_embeddingdescription_embeddingcommitr? exceptionrV )r r r r" r r r r( embedding_servicesessionr totalupdateditable embeddingrF s r r r s LK"I
MMO
JJszz+&
67 18 S
45
]g ]]>2668FKEz
89
]
JJw&89 : G %fa 0???u'<'<'< 1 F FuX]XmXm nI -6,<,<,>E)2;2B2B2DE/qLG~* (
\!AeWJ#GH 1
NN JJ6wiqwO P9 ]]<
8<$G sX 9H AG2H DoclingDocument.
Writes the document as markdown and/or JSON. No LLM inference.
By default uses the canonical `tuned-financial` configuration adopted in
QUE-218 (TableFormerMode.ACCURATE, DoclingParse backend, OCR enabled,
picture classification, page_batch_size=32) with `--device auto` resolving
to CUDA when available (QUE-219). Pass `--preset legacy` to revert to
the pre-QUE-218 behaviour, or `--device cpu` to force CPU on a GPU host.
r )AcceleratorDevice ImageRefModeparser_for_preset)accelerator_devicez Parsing (z , device=):
- JSON: r z - Confidence/provenance:
confidencez - Parsed cells: cells)r r z .docling.mdr z - Markdown: z
.docling.htmlTr split_page_viewz - HTML (split-page): N - Pages: - Tables: c 3 H # U H oR S :X d M Sv M g7f)ocrr2 N)verdict)rn r s r rp parse..^ s N 7199;MQQ 7s " "z, - Tables read by OCR (rendered as image): )%docling.datamodel.accelerator_optionsr docling_core.types.doc.baser quber.core.parsersr r rT rU parser r; r savesave_as_markdown save_as_htmlrv r r rw table_provenance)r r r r r r! r r r r parserrG rg r baseimage_ref_modeout artifactsmd_path html_path
ocr_tabless r r r s j H84
,V4F 8
$C JJ6()F83xjAB
\\#
FH 88D!*-N
Z C
KKT*
Z * 5 678
0<1H0IJK
' '(:';<= & &tfK 00G % %g. % I JJ y1 2 m 44I ! !)X\ ! ] JJ0 < =#
!&
JJS 0 123 JJ c(// 2 345N 7 7NNJ JJ=j\JK-
! s
CG33
HadesourcezaADE parse model (default: the generally-available dpt-2; dpt-3 preview names route to the v2 API)z--pagez&Submit only this 1-based page of a PDFpagec SSK Jn SSKJn U" XU=( d UUS9n[ R
" SUR SU 3U( a SU 3OS- 5 UR H n[ R
" S U 35 M [ R
" S
UR 35 [ R
" S UR 35 g )
a
Parse a PDF or image through Landing.AI ADE.
A standalone extraction path for what docling handles poorly: charts
(the parse reads a chart's plotted values off the page) and rasterized
pages with no native text layer. Writes the raw parse response and the
parse markdown, then a completion marker.
To read the charts in a document into its parse, use `quber figure`, which
scans only the pages that hold one.
r ) run_parse)
DEFAULT_MODEL)r r r z ADE parse (r z page r - r - Credits consumed: N)
quber.providers.landingr quber.providers.landing.clientr rT rU
model_versionr r credits)r
r r r r r summaryrd s r r r b s * 2<U=SmZ^_G JJW 2 2 33vh?VZVD6?`bcd!!
T$=! " JJW]]O,- JJ''89:r figurez--parsea The parse to enrich: a DoclingDocument JSON file, or a directory holding one, searched for .unified.json then .docling.json. The enriched parse is written back under the same filename, so pointing --output-dir at the directory it came from replaces it)requiredr zADE scan model. Defaults to the pinned dpt-3 version; 'dpt-3' selects that same pin, 'dpt-2' reverts to the previous line, and any exact ADE model name passes through as givenz--pages nominatedzPage breadth: 'nominated' scans the pages the parse points at; 'all' scans every page, for documents whose content the parse cannot be trusted to point atr r c t SSK JnJn SSKJnJnJn SSKJn
Ub UR 5 S:X a U
n[ U 5 n U R n U" X 5 u p[ U5 R 5 ( a [ U5 O[ U5 R nX S3- nSnUR 5 ( a8 [ R" " UR% SS 95 Vs0 s H nUS
U_M nnO![& R( " S UR* 5 [, R, " 5 n U" U U
U UUUUUU" 5 U" 5 S 9
nUR6 n[8 R: " SU 35 UR< H n[8 R: " SU 35 M [8 R: " SUR> 35 [8 R: " SUR@ SURB S35 [8 R: " SURD 35 [8 R: " S[G URH 5 35 [8 R: " S[G URJ 5 35 URL be [8 R: " S[G URL RL 5 SURL RN S URL RP S 35 [8 R: " S URR 35 URT H n[8 R: " S U 35 M U " UU S U UR< U[W S! URX 5 S5 UR> UR@ URB URD [G URH 5 [G URJ 5 URR URT [5 [, R, " 5 U-
S5 S".5 gs snf ! [. aO nU " UU S
U SU[1 U5 R2 SU 3[5 [, R, " 5 U-
S5 S.5 e SnAff = f)#a
Read the imagery in a document into its parse.
Three things a page prints are read by nothing else in the pipeline: the
values plotted in a chart, the content of any other picture, and a table
printed as an image, whose cells the parse read off the page itself with
nothing having checked it since. This reads all three through Landing.AI ADE.
The parse nominates the pages holding them, and every nominated page is
scanned; --pages all widens the run to every page of the document. Each page
is submitted whole, so a chart's title above it and the note at the foot
of the page come back with it, and a table comes back as a grid with a box on
every cell, which replaces the parse's reading of it.
Independent of the rest of the pipeline: in, the parse and the source
document; out, the same parse enriched, and the page scans. Writes the raw
response per page scanned, the figure records, the enriched parse under the
filename it was read from, then a completion marker. A page already scanned
is read from its stored response instead of being paid for again.
Run it after `quber fuse` so the enrichment lands in the document the rest
of the pipeline reads.
r )get_figure_value_readerget_scan_value_parser)
load_documentrun_figures_sync write_marker)SCAN_DEFAULT_MODELNzdpt-3 .cells.jsonutf-8encodingpage_noz-No {} beside the parse; figure values skipped)r r r cells_pages value_parser value_readerfailedr rc r2 )r r stager r@
total_secondsz
Figure scan: r z - Pages nominated: z - Pages scanned: (z submitted this run)z - Pages returning a figure: z - Figure records: z" - Tables read off a page image: z - Figure values: z
reconciled, z flagged)r z - ERROR: completec 3 ^ # U H# oR ( d M UR v M% g 7fNr )rn ss r rp figure.. s "Mi999199is --)r r r r r pages_nominated
pages_scannedpages_submittedpages_returning_a_figurefigure_recordsscanned_tables credit_usageerrorsr- )-quber.agents.figure_valuesr r quber.core.figures.orchestratorr r r! quber.core.figures.scanr" lowerr r; r is_dirparentr\ r loads read_textr warningrd timer? r __name__roundrX rT rU r r scanned submitted with_figuresrv figuresr values
reconciledflaggedr r; nextscans)r
r r r r r r r r r! r" r r r
parse_name parse_dir
cells_pathr( r startedrg excrX rd r1 s r r r s b Z]]:
}
0" V $E ::D(5H
$E{ 1 1 3 3U e9K9KIv[ 11JK04
:;O;OY`;O;a0bc0b1q|Q0b c F
XiikG! !#.002
6 **C JJ vh'(
T$=! ! JJ&s}}o67 JJ$S[[MCMM?BVWX JJ/0@0@/ABC JJ%c#++&6%789 JJ3C
4F3GHI
}}
!#fmm&:&:";!< =
(() v}}7L7L6MY
X
JJ' }56zz
[&' )) !"Mcii"Mt T "}} [[ "}}(+(8(8 !#++ . !&-- 0 KKjj "499;#8! list[ExtractedTable].
Engines:
- set-of-mark (default): the page image is the arbiter. The grid locator names
every table and its region; Camelot extracts each one constrained to its
box (so it can neither shatter one table into many nor merge many into
one); the grounded structure-correction LLM cleans headers/spans/merged
cells without altering a value. One ExtractedTable per visual table.
- camelot-llm (DEPRECATED): both Camelot flavors in parallel; per-candidate
is_table classification; LLM unify per page; structure-correction.
- correspondence (DEPRECATED): detector lists tables, Camelot chunks matched
by bbox overlap, completeness-audited, with recovery escalation.
Output is a JSON file of `ExtractedTable` Pydantic models with
provenance fields.
r MockClassifier)Backend
MockLLMClientget_llm_client MockUnifier) CamelotCorrespondenceExtractorCamelotLLMTableExtractorTableExtractorrY zu--no-llm is not supported with --engine set-of-mark; the grounded correction step is part of the set-of-mark capture.SetOfMarkExtractorr] llmzExtracting (SoM): rZ z--no-llm is not supported with --engine correspondence; the detector/correspondence/completeness flow is LLM-driven end to end.z Extracting (correspondence): F
llm_client
classifierunifierrun_llm_correctionr] z Extracting: T)ro rr r] extractz.tables.complete.jsonr+ rc r2 r r r, r@ r- Nc 3 # U H6 nUR ( d M UR R S :X d M2 Sv M8 g7f)detected_not_extractedr2 Nextraction_recordr rn ts r rp extract.. s< %
""
'(':':'A'AE]']
As A A A c 3 # U H5 oR ( d M UR R S :X d M1 Sv M7 g7f)
incompleter2 Nrw ry s r rp r{ s1
!!4!4A9L9L9S9SWc9cAAvs ?? ?r z (detected-not-extracted: z, incomplete: r
cell_flagswrite_artifacts .tables.json indentr r$ r% r .flags.jsonr - Flags: r. review item(s))render_annotated_pdfrender_flags_htmlrender_review_html .review.html
.boxed.pdfz - Review HTML: - Annotated PDF: .flags.html - Flags HTML (review queue): r/ )r r r flagsr- ))quber.agents.classifierra quber.agents.llm_clientrb rc rd quber.agents.unifierrf quber.core.extractorsrg rh ri rT
UsageError!quber.core.extractors.set_of_markrk r infor r; rE extract_tables_syncr? r
r rF rG rw rv quber.core.extractors.baser r
write_textr dumps
model_dumpr= quber.reviewr r r )'r r r[ r\ r] r r^ ra rb rc rd rf rg rh ri backend_litrk extractorr
r rT r, r rU rv r} r r r rh rz
flags_pathfr r r r
boxed_path
flags_htmls' r rs rs s v 7NN0 &1K
""F
I&3N;4OP (
34 # # ""V
3s{@[\ 3H:>? -$%' M $
l8*-.,%k2 #
l8*-.
h
'F ;;D iikG E..v6 !!!$ %
%
"
3v;- (((>'?~j\YZ
\
l3v;-01
6 v E
E$
$l 33I
F;FqALLNF;AsS^e
!
KK*YK0 1 dV;#7 7
%%djj%1P%Q!,,.%1PYZ&[fm%n k* RE
|CS TUdd .0@/A3D6Q]I^C^ _ 1&&PZH[B[ \
/ { ;< 1* >? .0@/A3D6Q\I]C] ^
KK"A* NO- %J &%& &kZ "499;#8! <
] f) *" I . ./r#7!&tyy{W' DualResult.
The visual flow looks at each rendered page and lets the grid locator
own the table count and identity; the Camelot flow pulls cell grids
straight from the PDF and tags each with an is_table verdict (metadata
only — it never drops a grid). The two run concurrently and neither is
matched to the other: this is the un-reconciled input for a later
cross-reference step. No detector, box repair, recovery, or escalation.
PDF_PATH may be a local path or an s3:// URI; s3 inputs materialize
through the persistent cache before extraction.
r )DualFlowExtractor)r] classify_camelotzDual extraction: Tr4 z
.dual.jsonr r r$ r% z - Visual tables: z Camelot grids: z - Per page (visual | camelot):z prc z | r N)quber.core.extractors.dualr r rT rU run_syncr r: r; r r r r r= visual_count_by_pagecamelot_count_by_pagesortedsetrv visualcamelotr )r r r] r r r
r rg r r rh visual_by_pagecamelot_by_pager r s r dualr sh . =
h
'F!cOLI JJ"8*-.
'F
z
CIIdTI* ;;Dj))I
F$5$5$73OZab002N 224O 3~&_)== >E JJ$S %7$88J3v~~K^J_`a JJ12
WQCr."4"4Q":!;3?R?RSTVW?X>YZ[ JJI;'(r z--artifacts-dira Cloud-style standalone fusion: load the document artifacts (.{docling,confidence,cells}.json) and the table artifact (.tables.json) from this directory and fuse only, skipping both engines. Without it, the document and table extractions run in-process.)r r r zHow pictures are written in the unified markdown/HTML. embedded (default) inlines them base64 so the files are self-contained; referenced writes a sidecar image folder and links it; placeholder drops them to ''.us Also write the human-reviewable control artifacts from both workflows: the table-side before/after HTML, annotated PDF (Set-of-Mark boxes on the fused tables), and — when any table carries flagged decisions — the flags HTML review queue, plus the document-side split-page HTML of the unified DoclingDocument. Rendered from the result already produced, no second pass.
artifacts_dirc
SSK Jn U" U5 n [ U 5 n
U
R n [ R " 5 n [ X+XXXFXq5
[ UU S3S
U [ [ R " 5 U -
S5 S .5 g ! [ aT n
[ UU S3SU S[ U
5 R SU
3[ [ R " 5 U -
S5 S.5 e S n
A
ff = f) aa
Fuse the document (docling) and table (Set-of-Mark/Camelot) extractions.
Two views of the same page enrich each other: the Camelot bodies move into
the docling spine, regions SoM merged are split, charts and image tables are
annotated, and tables docling found that SoM missed are surfaced explicitly.
Two control flows over one fusion core:
- Local (default): the document and table extractions run in-process, then
fuse.
- Standalone (--artifacts-dir): load the two extractions' artifacts from a
directory (as a cloud job would from S3) and fuse only, no engines.
Writes the unified DoclingDocument (markdown + JSON), the corrected
Set-of-Mark/Camelot tables (JSON), and a fusion report (per-region match
kinds and any surfaced errors). With --review, also writes the table-side
before/after HTML and annotated PDF and the document-side split-page HTML.
r )rd z.fuse.complete.jsonr+ fuserc r2 rt Nr/ )r r r- ) r rd r r; rE
run_fusionr? r
r rF rG )r r r r[ r] r! r r^ rd rm r
r fuse_startedrU s r r r s F 7
%C
h
'F ;;D
99;L=XsX^k &#$ "499; #=q A
f' (" I . ./r#7!&tyy{\'A1!E
s B
C
ACC r rm c
F SSK Jn
Jn U ( a SSKJn SSKJn
SSKJn U" U U S3U S3U S3U S 3/5 n[ R " S
U S U 35 U
R X5 n[ R " X S 3- R 5 5 Vs/ s H nU " S60 UD6PM
nn[ R " U " UUX#5 5 nOHSS KJn [ R " S
U S U 35 U
" [' U5 U" XcS9US9nUR) U5 nSSKJn SSKJn U" U5 nU" S UR2 5 5 n[ R " S[5 UR6 R8 5 35 [ R " S[5 UR8 5 35 [ R " S[; U5 35 UR< H n[ R> " SU 35 M [A U 5 nUU S3- n U RC [ RD " UR6 RG 5 SS9SS9 UU S 3- n UR6 RI U US 9 URJ bc UU S3- n U RC [ RD " URJ RL V s/ s H n U RO 5 PM sn S[P S 9SS9 UU S 3- n U RC [ RD " UR8 Vs/ s H nURO 5 PM snS[P S 9SS9 UU S 3- n!U!RC [ RD " [; U5 UR2 V"s/ s H n"U"RO 5 PM sn"UR< S!.S[P S 9SS9 [ R " S"U S#U 35 [ R " S$U 35 [ R " S%U! 35 SS&KJ)n# U#" UR8 5 URT - n$U$( ap UU S'3- n%U%RC [ RD " U$ V&s/ s H n&U&RO 5 PM sn&SS9SS9 [ R " S(U% S)[5 U$5 S*35 U( a SS+K+J,n'J-n(J.n) U)" UUR8 4/UU S,3- 5 n*U'" UR8 UUU S-3- 5 n+UU S.3- n,UR6 R_ U,US/S09 [ R " S1U* 35 [ R " S2U+ 35 [ R " S3U, 35 U(" UUR8 4/UU S43- 5 n-U-( a [ R " S5U- 35 S S S 5 g s snf s sn f s snf s sn"f s sn&f ! , ( d f g = f)7zSOne fusion run, artifacts in to artifacts out. The command wrapper owns the marker.r )DocumentFusionfuse_artifacts)r ) ParseResult)resolve_artifactsz
.docling.jsonz.confidence.jsonr# r zFusing from artifacts (r rj zFusing (rl )r r rm )Counterr c 3 8 # U H oR v M g 7fr1 )kind)rn ms r rp run_fusion.. s 3NqFFNs z - Unified document tables: z - Set-of-Mark/Camelot tables: z - Match kinds: r z
.unified.jsonr r r$ r% z .unified.mdr Nr z .fusion.json) match_kindsmatchesr; z
- Unified: z / r z - Report: r~ r r r. r r r r z
.unified.htmlTr z - Review HTML (tables): r z - Unified HTML (split-page): r r r )0quber.core.fusionr r r r r r quber.files.cacher r r loadr rB rC rW rX r rk parser_for_preset_lazy fuse_sync collectionsr r r r rv r r dictr; r@ r r r export_to_dictr r r r r= r
heading_flagsr r r r r ).r r r
rm r r! r] r r^ r r r r r r adirr rz
som_tablesrg rk fusionr r r kindsr1 r unified_json
unified_md
cells_jsonr tables_json report_jsonr r r r r r r r review_html boxed_pdf unified_htmlr s. r r r } s< A=27 ! &
&&()& $& %
-dV3xjAB ,37::tlF[?[>f>f>h3ij3ian)q)3i
j^E:v KLH hvhc(45 )&1(S:
!!&)#8!*-N 3FNN3 3E
KK/FOO4J4J0K/LMN
KK23v}}3E2FGH
KK#DK=12}} tC5\"
Z Cv] 33
6??+I+I+KTU Vah idV;//
(((O << # ${ 33J ! !
FLL4F4FG4FqALLN4FGPQ[^_
"
tfL 11 JJ
>
1
>qRU V
tfL 11 JJ#';8> G1 G $mm
mL>ZLAB l;-01 l;-0196==)F,@,@@ ${ 33J ! !$**e-Leallne-LUV"Wbi ! j KK+j\CJ>?V=VA V V:BV
V c SSK Jn U" U 5 $ )zABuild a parser, importing the GPU/ML stack only when fusion runs.r r )r r )r! r s r r r s 4 V $$r c SSK Jn SSKJn SSKJnJn SSKJn SSK J
nJ n SSK J
n
Un U" U 5 n U" U U" 5 U" 5 SUS 9n
[ R " S
U 35 U
R! [# U 5 5 n[$ R& " S S 9 n[# U5 nU" XS
[) U5 S9n/ n[+ USS9 H3 u nnUSUS S3- nUR- US5 UR/ U5 M5 [ R " S[1 U5 S35 U
" [# U 5 UUU 5 nSSS5 WR2 ( a [ R " S[1 UR4 5 S35 UR4 HO n[ R " SUR6 SUR8 S UR: S UR< S S 3 5 MQ O[ R " S 5 UR> ( aZ [ R " S![1 UR> 5 S35 UR> H n[ R " S"U 35 M gg! , ( d f GN/= f)#u
Run the Camelot-vs-LLM table-count validation hook.
Surfaces page-level mismatches between Camelot's detected count and
the LLM's image-based count. Per plan §7, mismatches are surfaced
explicitly; we do not silently judge which is correct.
r )convert_from_pathr` )rb rd re )rh ri )camelot_vs_llm_count_syncFrn zExtracting tables: zquber-validate-)prefixpng)r] r
output_folderr2 )startzpage-04dz.pngPNGzRunning validation across r Nz
MISMATCHES (z):z page z
: camelot=z llm=z (delta z+dr z
No mismatches detected.z
ERRORS (z ) pdf2imager r ra r rb rd r rf r rh ri quber.core.validater rT rU r r tempfileTemporaryDirectoryr= r r re rv has_mismatches
mismatchesr
camelot_count llm_countdeltar; )r r[ r] r ra rb rd rf rh ri r r rm r r tmptmp_pathimages page_imagesr imgr reportr rF s r validater s " ,6?0N=%0K
%C !9!#
!I
JJ$XJ/0
*
*4>
:F $ $,= >#9"8%sS[}]"$ a0FAsU1S'..A HHQ q ! 1
/K0@/AHI*4>6;PST
?
^C(9(9$:#;2>?""A JJ
1??2C5
U]^_^e^efh]iijk l #
./
}}
ZFMM 2 3267 A JJA3x %
? >s
BI
I z--hostz 127.0.0.1z&Interface to bind (default: 127.0.0.1)z--porti z Port to serve on (default: 8101)z--reloadz'Restart on source changes (development)hostportreloadc . SSK nUR SXUS9 g)z'Serve the answering playground web app.r Nzquber.playground.app:app)r r r )uvicornrX )r r r r s r
playgroundr D s KK*KPr rs c [ 5 g)z Main entry point for the CLI.N)r r r r mainr Z s Er __main__)<__doc__rW r rO r rE importlib.metadatar pathlibr typingr rT logurur quber.agents.factoryr r r r
r r quber.files.outputr r
quber.settingsr groupversion_optionr commandargumentoptionChoicer intr= boolrZ rz r r r r r r r r r r rs r r r r r r add_commandr rF )r s 0r r s
8t
& ; - . C ' gg ./ 0
ndH;_`
6
1ww
6 7 A i$de !#,I JTef ? @
>>> C=> # >
>
> >
> g M
l L>B|LC=|L
|L
|L
|L
|L
|L
|L |L
|L |L~%3 %
4!89 , - B gs3KL5!s 5!# 5!S 5! M
: 5!p h 2Z[ h35WX j$-VWQS Q QT Q X Y \ Q G $ i (
zF ' 7B +K~ 7s j9$j>k