Qj1<%SrSSKJr SSKrSSKrSSKJrJr SSKJr SSK J r J r SSK J r SSKJr SS KJr \"5R$R&rS \S '\S - rS \S 'SSjrSSjrSSjrSS.SSjjrSSjrSSjr\S:Xa\"5 gg)a#Where a document's files live. Every file the playground keeps for a document is named by the document's storage key and sits in the local data directory: the served PDF, the parse and figure artifacts the ingest read, and the upload workdir. On the developer host that directory is the only copy. Hosted, the local directory is scratch that a task replacement erases, and the durable copy is an ``s3://`` prefix from settings holding one ``/`` prefix per document with the same file names inside it. Two movements keep the two in step: ``publish`` copies files up after every stage of an upload, and ``local_pdf`` fetches the served PDF back down when a task no longer has it. Nothing here ever deletes from the prefix; removing a document from the library removes its database rows and local files only, so a later upload of the same bytes finds every artifact still there. Run as a module, it publishes the whole current library once, which is how the host's documents were moved into the bucket: uv run python -m quber.playground.storage ) annotationsN)datetimetimezone)Path)IterableOptional)S3Path)logger) get_settingsrDATA_DIRuploadsUPLOADSc[5RRnU(dg[UR S5S-U-S-5$)zCThe document's prefix in the bucket, or None on the developer host.N/)r playground artifacts_urir rstrip)doc_keyroots 6/home/mande/repo/quber/src/quber/playground/storage.pyprefixr(s> > $ $ 2 2D  $++c"S(72S8 99c[U5nUcgSnUH;nUR5(dMX$R- RUSS9 US- nM= U(a[R "SX0U5 U$)zCopy the given local files under the document's prefix. Returns how many were copied; zero, without touching anything, on the developer host or when none of the paths exist.rT)force_overwrite_to_cloudz!published {} file(s) for {} to {})ris_filename upload_fromr info)rpathsdestcopiedpaths rpublishr$0sp '?D | F||~~  &&td&K!    7$O Mrc^UR5(dg[T[U4SjUR5555$)z:Copy every file the upload workdir holds for the document.rc3j># UH(oRRT5(dM$Uv M* g7f)N)r startswith).0prs r "publish_workdir..Fs%"^.?66CTCTU\C]11.?s#3 3)is_dirr$sortediterdir)rworkdirs` rpublish_workdirr0Bs2 >>   7F"^goo.?"^^ __r) source_uricUUUU[R"[R5R SS9S.n[ US3- nUR RSSS9 UR[R"USS95 U$) zRecord where the document came from beside its PDF: the name it was uploaded under, its full hash, the S3 object it was taken from if any, and when. The key carries none of this, so the prefix says it instead.seconds)timespec)rfilenamesha256r1 uploaded_at .source.jsonTparentsexist_ok)indent) rnowrutc isoformatrparentmkdir write_textjsondumps)rr5 content_hashr1manifestr#s rwrite_source_manifestrHIs{ ||HLL1;;Y;O H y - -DKKdT2OODJJx23 Krc[US3- [US3- 4HnUR5(dMUs $ [U5nUcgX S3- nUR 5(dg[US3- nUR R SSS9 [R"SURU5 URU5 U$)zuThe served PDF as a local path, fetched from the bucket when this host no longer has it. None when it exists nowhere.z.pdfNTr9zfetching {} from {}) r rrrexistsrArBr rr download_to)r candidateremotesourcetargets r local_pdfrP\sG9D!117y=M3MN      OG_F ~  & &F ==?? 7)4( (F MMt4 KK%v{{F; v Mrc l^SSKJn [S5c[R"S5 UR 5nUR S5R5nSSS5 SnWHumpE[[RTS355n[U4SjU55(dUR[TXES 95 [TU5[T[ TS 3- 5-nX7- n["R$"S TU5 M ["R&"S U[)U55 g!,(df  N=f) zPublish every document the database knows about from the local data directory: the served PDF and the artifacts beside it, plus the upload workdir when this host still has it.r)dbprobeNz@QUBER_PLAYGROUND_ARTIFACTS_URI is not set; nothing to publish tozPSELECT doc_key, filename, content_hash FROM ade_playground.documents ORDER BY idz.*c3F># UHoRTS3:Hv M g7f)r8N)r)r(frs rr*"publish_library..~sEu!66y 55us!)r5rFz -artifactsz{}: {} file(s)z'published {} file(s) for {} document(s))quber.playgroundrRrsysexitconnectexecutefetchallr-r globanyappendrHr$r0rr rsuccesslen) rRconnrowstotalr5rFfilesr"rs @rpublish_libraryrfos$ g ST || ^ (*   E+/'x}}y^45EuEEE LL.we f%(?7GQXPYYcNdDd+ee  $gv6 ,0 NNrws,# '%'**33$39$$:$`TX "25CP &&P. zr