ڰjUSrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKJ r SSK J r \ "5r Sr SrSrSrS rS rS r"S S 5rSr/rSrSrSr\R4"\5 g)aDirectory-mirror warm cache between local directories and a network volume. ``VolumeCache`` keeps a browsable mirror of one or more local directories on a mounted network volume, and reconciles the two directions on demand: - ``hydrate()``: copy files that are missing or newer on the volume mirror into the container (used on cold start, before the cache is populated). - ``sync()``: copy files that are missing or newer in the container onto the volume mirror (used after the cache has been populated/updated). Transport adapts to the tree's shape: files below 256 KiB are packed into a single ``small.tar`` on the volume (collapsing per-file metadata round-trips), larger files are copied unpacked into ``big/`` across a thread pool, and a versioned ``manifest.json`` (written last) records the layout. Large-file transfers stay incremental (size/mtime diff); the small-file archive is re-packed whole when any small file changes. Packing uses the ``tar`` binary when present (else stdlib ``tarfile``); extraction always uses ``tarfile``, validating every member against the configured dirs. Best-effort: any failure degrades to a cold worker and never raises unless ``best_effort=False``. N)ThreadPoolExecutor) RunPodLoggerg@g>@iz manifest.jsonz small.tarbigc\rSrSrSr\R S-\R -\R S-\R -4rSSSSS.S jr\ S 5r \ S 5r \ S 5r \ S 5r \ S5rSrSrSrSrSrSrSr\S5rSrSrSr\S5rSrSrSrSr\S5r S r!S!r"SS".S#jr#S$r$S%r%S&r&S'r'g)( VolumeCache,aWarm-cache local directories across serverless workers via a network volume. Maintains a mirror of ``dirs`` at ``{volume_path}/.cache/{namespace}``. ``hydrate()`` reconciles volume -> container; ``sync()`` reconciles container -> volume. Used as a context manager, the object is itself the "warm cache" closure: hydrate on enter, sync (in the background by default) on exit. Requires a network volume mounted at ``volume_path`` (default ``/runpod-volume``) and a non-empty ``namespace`` (default ``RUNPOD_ENDPOINT_ID``); otherwise ``available`` is False and all operations are no-ops. Args: dirs: Local directories to cache (e.g. a model cache like ``HF_HOME``). namespace: Isolation key for the on-volume mirror. Must be a single safe path component. Defaults to ``RUNPOD_ENDPOINT_ID``. volume_path: Network-volume mount point. Defaults to ``/runpod-volume``. best_effort: When True (default), swallow and log errors instead of raising. max_workers: Thread count for parallel copy of large files. Defaults to ``min(32, (os.cpu_count() or 4) * 4)`` (I/O-bound, so oversubscribing the CPU count is fine). Example: >>> with VolumeCache(dirs=["/root/.cache/huggingface"]): ... model = load_model() # downloads land in the cached dir refsz .no_existNz/runpod-volumeT) namespace volume_path best_effort max_workersc [U[[[R45(aU/nUVs/sHJn[R R [R"[R"U555PML snUl U=(d( [RRS5=(d SUl UR(a[R RUR5(dN[RUR;d0SUR;d SUR;dURS;a[SUR<35e[R"[R"U55UlX@lUbUS:a[SU<35eU=(d+ [%S [R&"5=(d S S -5Ulgs snf) NRUNPOD_ENDPOINT_ID/\).z..z4namespace must be a single safe path component, got rz,max_workers must be a positive integer, got ) isinstancestrbytesosPathLikepathrealpathfsdecodefspath_dirsenvironget _namespaceisabssep ValueError _volume_path _best_effortmin cpu_count _max_workers)selfdirsr r r rds d/home/mande/repo/quber/.venv/lib/python3.13/site-packages/runpod/serverless/utils/rp_volume_cache.py__init__VolumeCache.__init__LsY dS%5 6 66DLPP4abgg&&r{{299Q<'@A4P #Qrzz~~6J'KQr ?? GGMM$// * *vv(doo%t&+-FtFYZ KK +(>?'  "{QKK?[\ \'M3rBLLN4Ga13L+M/QsAG.cl[RRURSUR5$)Nz.cache)rrjoinr'r#r,s r/ _mirror_rootVolumeCache._mirror_rootos#ww||D--xIIc^[RRUR[5$N)rrr3r5_MANIFEST_NAMEr4s r/_manifest_pathVolumeCache._manifest_pathssww||D--~>>r7c^[RRUR[5$r9)rrr3r5_SMALL_ARCHIVE_NAMEr4s r/_small_archive_pathVolumeCache._small_archive_pathwsww||D--/BCCr7c^[RRUR[5$r9)rrr3r5 _BIG_SUBDIRr4s r/ _big_rootVolumeCache._big_root{sww||D--{;;r7c[UR5=(a) [RR UR 5$)zATrue iff volume_path is a mounted dir AND namespace is non-empty.)boolr#rrisdirr'r4s r/ availableVolumeCache.availables+DOO$It7H7H)IIr7c#^# [R"U5Hup#nUHn[RRX%5mUR S5(dUR S5(aMP[ U4SjUR 55(aMv[RRT5(aMTv M M g![a Mf=f7f)Nz.lock .rpvc.tmpc3,># UH oT;v M g7fr9).0subrs r/ *VolumeCache._iter_files..sG.Fsd{.Fs) rwalkrr3endswithany_EXCLUDE_SUBSTRINGSislinkOSError)r,rootdirpathr filesnamers @r/ _iter_filesVolumeCache._iter_filess%'WWT] !GEww||G2==))T]];-G-GGd.F.FGGGww~~d++ , &3s*BC$C =C CCCCc//p2UHAn[R"U5RnU[:aUOUR U5 MC X#4$![a MUf=fr9)rstatst_sizerW_SMALL_FILE_THRESHOLDappend)r,pathssmallrpsizes r/ _partitionVolumeCache._partitionscsA wwqz))22U C CA F  z  s A AAc[R"U5nXRURS.$![a gf=f)Nrrfmtime)rr_rWr`st_mtime)r,rsts r/ _file_metaVolumeCache._file_metas= Bjj2;;GG  s 1 >>c[R"URSS9 [[UUS.nUR S[R "5S[R"5S3n[US5n[R"X55 SSS5 [R"X@R 5 g!,(df  N/=f)NTexist_ok)version thresholdrdrrrKw) rmakedirsr5_FORMAT_VERSIONrar;getpid threading get_identopenjsondumpreplace)r, small_metabig_metapayloadtmpfhs r/_write_manifestVolumeCache._write_manifests D%%5&.   $$%Qryy{m1Y5H5H5J4K9U #s^r IIg " 3++,^s 6B66 Cc[UR5n[R"U5nSSS5 [ W[5(aURS5[:wagU$!,(df  N?=f![[ 4a gf=f)Nrs) r{r;r|loadrWr&rdictr"rw)r,robjs r/_read_manifestVolumeCache._read_manifestsq d))*biim+#t$$ (:o(M +*$  s-A8A'A8' A51A85A88B  B c[R"US5nURUS:H=(a URUS[ - :$![a gf=f)NrFrfrkrr_rWr`rl_MTIME_TOLERANCE)r,metarms r/_meta_satisfied_by_local$VolumeCache._meta_satisfied_by_localsZ f&BzzT&\)]bkkT']M]=].]]  sA AAcU(aURU/5O/Vs0sHoDSU_M nn/nUHLnURUS5nUb!USUS:wdUSUS[-:dM;URU5 MN U$s snf)Nrrfrk)r"rrb) r,metasmanifestkeyepriorchangedmres r/_changed_vs_manifest VolumeCache._changed_vs_manifestsAI S"(=r(QS(Q!6A(QSA !F)$AyAfI62aj1W:P`C`6`q! TsBc[R"U5n[R"U5nURUR:g=(d URUR[ -:$![a gf=f![a gf=f)NFTr)src_pathdst_pathsr.s r/ _needs_copyVolumeCache._needs_copys| !A !AyyAII%SajjCS6S)SS     s"A/A?/ A<;A<? B  B c^[RRU5m[U4SjUR55$)Nc3># UH3nTU:H=(d" TRU[R-5v M5 g7fr9) startswithrr%)rNr.targets r/rP,VolumeCache._is_safe_dest..s0TA6Q;?&"3"3AJ"??s;>)rrrrTr )r,dst_absrs @r/ _is_safe_destVolumeCache._is_safe_dests,!!'*TTTTr7c &US[R"5S[R"5S3n[R"[R R U5SS9 [R"X5 [R"X25 g![atn[RSUSUSU35 [R RU5(a[R"U5 O![a Of=fSnAgSnAff=f) NrrKTrqzVolumeCache: skip z -> : F)rrxryrzrvrdirnameshutilcopy2r~rWlogdebugexistsremove)r,srcdstrexcs r/ _copy_fileVolumeCache._copy_filesQryy{m1Y%8%8%:$;9E  KK,t < LL " JJs   II*3%tC53%@ A 77>>#&&IIcN   s<AB DD ;:C65D 6 DD DD  Dc ^UVVs/sH up#TRX#5(dMX#4PM" nnnUVVs1sHup%UiM snnUVVs1sHup%UiM snn- nU(d [5U4$[TRS9nUR U4SjU5n[ XH5VVV s1sHuup%o(dMUiM n nnn SSS5 X4$s snnfs snnfs snnfs sn nnf!,(df  W U4$=f)aCopy every (src, dst) that needs it, in parallel. Returns ``(copied, current)`` sets of source paths: - ``copied`` -- sources transferred successfully this call. - ``current`` -- sources skipped because the destination was already up to date. Their union is the set of sources the destination now holds a current copy of; a source in neither failed to copy. Callers that record a "mirror complete" manifest MUST include only sources in that union, or a failed copy would be marked present and later served stale. copy2 + atomic os.replace means a source in ``copied`` matches its destination at that instant, so no re-stat is needed to confirm it. )rc4>TRUSUS5$)Nrr)r)sdr,s r/,VolumeCache._copy_parallel.. s$//"Q%A*Gr7N)rsetrr+mapzip) r,pairsrr.todo_currentpoolresultsokcopieds ` r/_copy_parallelVolumeCache._copy_parallels$)C541D,<,1*>>5'> ! D,=,= >$hhGNG*-d*<C*D ? > ws9CCC C &C-,C& ?C& C-&C-- C>c[RRU5n[RRU5nX2:H=(d" URU[R-5$)z?True iff ``path`` resolves inside ``base`` (symlinks resolved).)rrrrr%)baser base_real path_reals r/_withinVolumeCache._within sNGG$$T* GG$$T* %Q)=)=i"&&>P)QQr7cU"5$![a6nUR(de[RSU35 UsSnA$SnAff=f)NzVolumeCache operation failed: ) Exceptionr(rwarn)r,fndefaultrs r/_guardVolumeCache._guardsC 4K $$ HH5cU; <N  s A +AA A c.[R"S5$)Ntar)rwhichr4s r/ _tar_binaryVolumeCache._tar_binarys||E""r7c \[R"URSS9 URS[R"5S[ R "5S3nUVs/sH#n[RRUS5PM% nnUR5(aUS3n[US5nURSRU55 SSS5 [R"S S SS S S USU/ S[R[R0[R ESS0ES9 [#U5 OD[$R"US5n['X5Hup8UR)X8SS9 M SSS5 [R*"X R5 gs snf!,(df  N=f![#U5 f=f!,(df  NU=f![,[R.[$R04a-n [2R5SU 35 [#U5 Sn A gSn A ff=f)NTrqrrKrz.listru rz-Cz-cz--no-recursionz-fz-TCOPYFILE_DISABLE1)checkstdoutstderrenvF)arcname recursivez'VolumeCache: pack small bucket failed: )rrvr5r?rxryrzrrelpathrr{writer3 subprocessrunDEVNULLr!_silent_removetarfileraddr~rWSubprocessErrorTarErrorrr) r,rZrfrelslistingrtfrelrs r/ _pack_smallVolumeCache._pack_small s D%%5))*!BIIK=):M:M:O9PPYZ167A3'7 !! E- ,gs+r41,NNc41A4dT[\")11)11CrzzC+=sC #7+\\#s+r"%e"2q?#3, JJs44 5-8 ,+#7+++ 33W5E5EF  II?uE F 3  sg*F G' F43!F#AF4-#G%G5(G# F1-F44 GG GG)H+>#H&&H+c[R"UR5nSnUR 5nUHnUR5(dM[RRSUR5nURU5(dM\XeRUR S.nUR#U5(aMUR%X5 US- nM UR5 U(a[ R'SUS 35 U$![[R4a gf=f![[R4aAn[ RSURSU35 UsSnAUR5 $SnAff=f![[R4a1n[ RSURSU35 SnAGMSnAff=f!UR5 f=f) NrzVolumeCache: corrupt archive rrrjrzVolumeCache: skip extract zVolumeCache: extracted z small file(s))rr{r?rWr getmembersrrcloseisfilerrr3r[rrfrkr_extract_memberinfo)r,r extractedmembersrmemberrrs r/_extract_smallVolumeCache._extract_small?s d667B   !--/"}}ggll3 4))#.. #[[6<<P0066Q((4NI"& HHJ  HH.ykH I?))*   W--. ! 9$:R:R9SSUVYUZ[\  ( HHJ- !& !1!12QII :6;;-r#OPPQ HHJsp DD2BGFGD/.D/2F  &F2F 3GF  GG*%GGGGG0cv[[S5(aURUSSS9 gURUS5 g)N data_filterrdata)filter)hasattrrextract)rrs r/rVolumeCache._extract_membercs1 7M * * JJvs6J 2 JJvs #r7c^UR(dgURURS5$)zReconcile volume mirror -> container. Copy files missing or newer in the container. Returns the number of files copied. No-op (0) if unavailable. r)rHr _do_hydrater4s r/hydrateVolumeCache.hydrateps% ~~{{4++Q//r7cUR5nUcgSnURS5(aX R5- n/nURS/5Hn[RR USS5n[RR SU5n[RR URU5nURU5(dMURURU5(dMURXv45 M URU5upU[U5- nU(a%[RSUSUR35 U$)NrrdrrrzVolumeCache: hydrated z file(s) from )rr"rrrrr3rCrrrbrlenrrr5) r,rrestored big_pairsentryrrrr_currents r/r VolumeCache._do_hydratezs&&(   << ++- -H \\%,E''//%-5C'',,sC(C'',,t~~s3C!!#&&4<<+L+L  #,- ..y9CK  HH-hZ~dFWFWEXY Zr7 backgroundc^TR(dgU(a5[R"U4SjSS9n[U5 UR 5 gTR TR S5 g)a'Reconcile container -> volume mirror. Copy files missing or newer on the volume. When ``background=True`` (default), run on a daemon thread and return immediately; a process-exit hook joins outstanding syncs so short-lived processes still complete. When False, run inline and return when done. Nc<>TRTRS5$)Nr)r_do_syncr4sr/r"VolumeCache.sync..s DMM10Mr7T)rdaemonr)rHryThread_register_pendingstartrr)r,rts` r/syncVolumeCache.syncsI~~    (MVZ[A a GGI KK q )r7c 0^/nTRHIn[RRU5(dM)UR TR U55 MK TR U5up4TR5nU4SjU5Vs/sH of(dM UPM nnSnU(aTRXuS5n U(aURS/5O/V s1sHoSiM n n [XVs1sHofSiM sn- 5n U (dU (aNTRUVs/sHofSPM sn5(a [U 5nOXGVs/sHofSPM sn-n/nU(d[TR5 UV s/sHLo[RRTR [RR#U S554PMN nn TR%U5unnUU-nU4SjU5Vs/sHof(dM USU;dMUPM nn[U5nTR'UU5 UU-nU(a%[(R+SUSTR,35 U$s snfs sn fs snfs snfs snfs sn fs snf) Nc3F># UHnTRU5v M g7fr9rnrNrr,s r/rP'VolumeCache._do_sync..s!Jk$//!"4"4k!rrdrrc3F># UHnTRU5v M g7fr9r#r$s r/rPr%s>Iq**Ir&zVolumeCache: synced z file(s) to )r rrrGextendr\rgrrr"rFrrrr?r3rCrrrrrr5)r, all_filesrX small_files big_filesrrrsmall_transferredrr prior_smalldeletedrrrrmirroredr big_copiedtotals` r/rVolumeCache._do_syncsR JJDww}}T""  !1!1$!78"&!; &&(!Jk!JP!JAaa!J P // gNGLTx||GR/HZ\/\^/\V9/\K^;Z)HZF)Z)HHIG' ## $C 1vY $CDD(+G %!* ,K 1vY ,K KI!#J 433 4YbcYbTUdnnbggooa6MNOYb c--i8 G#>I> >!!A& U]H]A>  [  Z2..  HH+E7,t?P?P>QR S UQ_)H%D -Ld s= I5I5I:5I? )JJ AJ JJJc&UR5 U$r9)r r4s r/ __enter__VolumeCache.__enter__s  r7c"URSS9 g)NTr)r)r,rs r/__exit__VolumeCache.__exit__s T "r7)r(r r+r#r')(__name__ __module__ __qualname____firstlineno____doc__rr%rUr0propertyr5r;r?rCrHr\rgrnrrrr staticmethodrrrrrrrrrrr r rrr4r7__static_attributes__rMr7r/rr,sd:66F?RVV3RVVk5IBFF5RS"&3CQUcg!NFJJ??DD<<JJ H - ^ T TU"4RR #>"H$$0."&*"2pr7rcR[R"U5 g![a gf=fr9)rrrW)rs r/rrs$  $    s  &&c[Vs/sHoR5(dMUPM sn[SS&[RU5 gs snfr9)_pending_syncsis_aliverb)threadrs r/rrs5$2CNqjjlNCN1&!Ds A A c [[5HLnUR[S9 UR 5(a [ R S[SS35 MLMN [R5 g![a Mrf=f)N)timeoutz3VolumeCache: background sync did not finish within z.0fz"s at exit; cache may be incomplete) listrCr3_JOIN_TIMEOUT_SECONDSrDrrrclear)rs r/_join_pending_syncsrKsw . !  FF0F 1zz||I,S11SU"   sAA55 BBc,[R5 gr9)rCrJrMr7r/_reset_pending_for_testrM sr7)r=atexitr|rrrrryconcurrent.futuresr#runpod.serverless.modules.rp_loggerrrrrIrarwr:r>rBrrrCrrKrMregisterrMr7r/rRs* 1<n" ! ttx  " #$r7