Ei}>(SSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJ r SSK J r J r J r SSKJr SSKJr SSKJr SS KJrJr \R."\5r\R4"S S 9"S S 55r\R4"S S 9"SS55r\R4"S S 9"SS55r\R4"S S 9"SS55r\R4"S S 9"SS55r\R4"S S 9"SS55r "SS5r!g)N)Path)Optional)counters dynamo_timedset_feature_use)justknobs_check)FileLock)triton_cache_dir) _IS_WINDOWSGPU_KERNEL_BIN_EXTST)frozenc8\rSrSr%Sr\\S'\\S'\\S'Srg)TritonBundleEntryz When we have compiled a triton kernel, we take note of that kernel by its triton generated hash, its device, and where this kernel is located. This is the minimum information we can use to later retrieve this kernel from file system. kernel_hashdevice directoryN) __name__ __module__ __qualname____firstlineno____doc__str__annotations__int__static_attributes__r[/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/_inductor/triton_bundler.pyrrs KNrrcN\rSrSr%Sr\\S'\R"SS9r \ \S'Sr g) TritonKernelArtifact#ze Artifact for an individual kernel converted to bytes. Bytes could be a cubin, json, ttir, or ttgir. filenameF)reprpayloadrN) rrrrrrr dataclassesfieldr&bytesrrrr r"r"#s# M &&E2GU2rr"c8\rSrSr%Sr\\S'\\S'S\S'Srg) StaticallyLaunchedAutotuner.a= Represents a statically compiled CachingAutotuner object that we can save directly in the cache. A CachingAutotuner is made up of a list of StaticTritonCompileResults, each of which uses the cubin from a TritonKernelArtifact. Statically saved here have their cubin files saved by a corresponding TritonBundleEntry. cache_key kernel_nameCachingAutotunerkernelrN)rrrrrrrrrrr r+r+.sN rr+c>\rSrSr%Sr\\S'\\S'\\ \S'Sr g)TritonKernelArtifacts=z2 Collection of artifacts for a particular kernel. rr artifactsrN) rrrrrrrrlistr"rrrr r2r2=s  K())rr2c:\rSrSr%Sr\\\S'\\\S'Srg)TritonBundlerMetadataHz# Metadata used for instrumentation cached_kernel_names statically_launched_kernel_namesrN) rrrrrr5rrrrrr r7r7Hsc"&*3i/rr7c:\rSrSr%Sr\\\S'\\\S'Sr g) TritonBundleRz/ Serializable bundle to save into FXGraphCache kernel_artifactsstatic_autotunersrN) rrrrrr5r2rr+rrrr r<r<Rs!011788rr<c\rSrSr%SrSr\\\\ S'Sr \\\ \ S'Sr \ \ S'\S\4S j5r\SS j5r\SS j5r\S \S \SS4Sj5r\S\SSSS4Sj5r\S\\\ \\44Sj5r\S\\\ S\\4Sj5r\S\\\\44Sj5r\S\S\\4Sj5rSrg) TritonBundler\a Lightweight Triton Kernel bundler that notes each time we compile a triton kernel. When collect is called, converts all the previously noted kernels and their artifacts into a structured bytes blob, and later when write is called it writes this structured blob back to file system. Intended Life cycle: - TritonBundler.begin_compile is called when we start compiling in Inductor - TritonBundler.put is called each time a Triton Kernel is compiled - TritonBundler.collect is called when a cache entry is being generated - TritonBundler.end_compile is called to indicate bundling is completed, collect will execute this function as well. - TritonBundler.read_and_emit is called when a cache entry is read N_entries_static_autotunerss [REPLACE]_REPLACE_BYTESreturncSSKJn UR(agUR=nbU$UR 5(dg[ S5$)NrconfigFz9pytorch/remote_cache:bundle_triton_into_fx_graph_cache_v2)torch._inductorrIforce_disable_caches!bundle_triton_into_fx_graph_cache is_fbcoder)rIbs r is_enabledTritonBundler.is_enabledssI*  & &99 9A FH!! G  rc[R5(dg[RS5 URbe/Ul/Ulg)zh Initializes the TritonBundler. The current TritonBundler bundle is finalized by TritonBundler.collect. Nz%TritonBundler.begin_compile is called)rArOlogdebugrCrDclss r begin_compileTritonBundler.begin_compilesB ''))  9:||### !#rcJ[RS5 SUlSUlg)z\ Finalizes the TritonBundler. If collect is not yet called, it discards the current bundle. z#TritonBundler.end_compile is calledN)rRrSrCrDrTs r end_compileTritonBundler.end_compiles 78 !%rrrc lUR=nb%UR[X[U555 gg)zo Lazily observes that we have seen a Triton kernel compilation. Remembers it for when collect is later called. N)rCappendrr )rUrrentriess r putTritonBundler.puts4 || #G 0 NN!+7G7OP  1rkeyr0r/c ^SSKJn UR(deUR=nbUR 5n[ R "U5nUR5 SUlUR[UURRSS5U55 URU5 gg)NrrHr.unknown_kernel)rJrIuse_static_triton_launcherrDprepare_for_picklecopydeepcopyprepare_for_caching_reload_kernelr\r+ inductor_metagetrestore_after_unpickle)rUr`r0rIr] old_values new_kernels r put_static_autotuner"TritonBundler.put_static_autotuners*0000-- -G : 224Jv.J  * * ,(,J % NN+,,00@PQ   ) )* 5# ;rcUR(d//4$[RS[UR55 URVs/sHoRPM nn[ SS==S- ss'URU4$s snf)Nz1Saving %d statically launchable CachingAutotunersinductor$triton_bundler_save_static_autotunerr )rDrRinfolenr.r)rUistatic_autotuner_namess r collect_static_autotuners'TritonBundler.collect_static_autotunerss%%r6M HHCC**+ >A=S=S%T=Smm=S "%T Z !G HA M H))+AA A&Us Br?c <U(d/$SSKJn SSKJn [R S[ U55 /n[S5 UHnURRHnUR5 M U"UR5URUR'[ S S ==S - ss'UR#UR5 M S S S 5 U$![a# [RSURSS9 Mf=f!,(df  U$=f) z^ Load statically launchable CachingAutotuners into async_compile.CompiledTritonKernels cache. r)CompiledTritonKernels)StaticAutotunerFuturez+Loading %d statically launchable autotunersz+TritonBundler.load_cached_static_autotunersz>Failed to reload cubin file statically launchable autotuner %sTexc_inforq$triton_bundler_load_static_autotunerr N)torch._inductor.async_compilerztorch._inductor.codecacher{rRrsrtrr0compile_resultsreload_cubin_path RuntimeErrorwarningr._cacher-rr\)rUr?rzr{ kernel_namesresultcompile_results r load_autotunersTritonBundler.load_autotunerss!IGC >DU@VW G H+ *0--*G*G&88:+HBWMMB%,,V-=-=>$%KLPQQL##F$6$67),I,!$KKX**!%    I H,s1D  -C6AD )D D D  D  Dc SSKJn [R5(d*UR 5 [ SS5 [ //5S4$[ SS5 [SSS9 URnUGb/n/nUGHAn/n[RRURUR5n[RRU5(dM`[R"U5GHn[RRXx5n [RR!U 5(de[#U S 5n U R%5n U R'S 5(aw[R(U ;a0[*R-S [R(U 5 [/S 5eU R1[2R5U5[R(5n UR7[9X55 SSS5 [:S S==S- ss'[RRAU5Sn U [BRD"5;dGMhUR7[GU5RH5 GM U(dGMUR7[KURURLU55 GMD URN(aURQ5upO/n /nUR 5 [ X=5[SXN54sSSS5 $[ //5S4sSSS5 $!,(df  GN7=f![<a [*R?SSS9 GNGf=f!,(df  g=f)z This is the main function called when a cache write happens. This function converts all the previously remembered kernels into bundled format so that it can be written into a cache entry. This function also finalizes the current bundle. rrHtriton_bundlingFNTzTritonBundler.collectr`log_pt2_compile_eventrb.jsonz'Bundle contains illegal %s, payload: %szBundle contains illegal bytesrqtriton_bundler_save_kernelr zfailed to collect triton kernelr|)*rJrIrArOrYrr<rrCospathjoinrrexistslistdirisfileopenreadendswithrErRrAssertionErrorreplacerencoder\r"r ExceptionrSsplitextr valuesrstemr2rrcrwr7)rUrIr]rrentryr4rr$filepathfiler& extensionr?static_kernel_namess r collectTritonBundler.collects +'')) OO  -u 5B'- -)40 5T RllG"68*, $E<>I77<<9J9JKD77>>$// $&JJt$4#%77<<#?X#%77>>(#;#;;#;!%h!5*.))+#+#4#4W#=#='4'C'Cw'N(+ ,U,9,H,H,3)* /=,K/*)*/6oo(+ 4(8-:V:V/&G!* 0 0$8$K!"#"6(%Z01MNRSSN%'GG$4$4X$>q$A $(;(B(B(DD)//X0C0CD?%5@!y 1 % 1 1 % )M%Z44557;%':)+%*,'!#F>@U AuS Rz B'-{S R"6!5* )XII&GRVIWXGS Rs] B0M"2L=B8L+ ;L=9M".M"BM" M"+ L: 5L==M M"M M"" M0bundlec SSKJn [R5(dg[ SSS9 /nUR GHn[ UR5n[RRXCR5n[RRU5(a;[[R"U55S:wa[R!SU5 M[#U5R%SSS9 ['[(R*"55n[RRUS U35n[R,"U5 UR.GH<n[RRXxR05n [3U S 5n UR4n UR0R7S 5(a3U R9[R:[&R=U55n U R?U 5 SSS5 [@S S ==S- ss'[RRCUR05Sn U [DRF"5;dGMURI[#UR05RJ5 GM? [L(aj[OUS-5 [RRU5(a[PRR"U5 [R8"Xu5 SSS5 GM[R8"Xu5 GM URX(a [R[UR\5n O/n [_X-5sSSS5 $!,(df  GNv=f!,(df  GMX=f![Ta [RWSU5 GMf=f!,(df  g=f)a This is the main function called when a cache read happens. This function converts the bundled format back into individual files and writes them to the filesystem. NOTE: When we are writing to the filesystem, we assume exclusive access to the target directory. This means that if the target folder already exists and is non-empty, we bail out. Exclusive access means that no other process should be writing to or reading from the target directory. rrHNzTritonBundler.read_and_emitTrz8Bailing out TritonBundler.read_and_emit, %s is non empty)parentsexist_okztmp.wbrrq#triton_bundler_read_and_emit_kernelr z.lockz%Directory %s is not empty - skipping!)0rJrIrArOrr>r rrrrrrrtrrRrSrmkdirruuiduuid4makedirsr4r$rr&rrrErwriterrr rr\rr r shutilrmtreeOSErrorrrcrr?r7)rrIrr4basedirrrnd_idtmp_dirartifactrrr&rrs r read_and_emitTritonBundler.read_and_emitFs +'')) -T ')L#44 *9+;+;<GGLL2G2GH 77>>),,RZZ 5J1Kq1PIIR!W ##D4#@TZZ\*'',,w$vh@ G$ ) 3 3H!ww||G5F5FGHh-"*"2"2#,,55g>>&-oo - < S'G 7+ .Z()NOSTTO " 0 01B1B CA FI $7$>$>$@@%++D1B1B,C,H,HI!4 ;!)g"5677>>)44"MM)4 7676 V 76Y5`00&3&C&C,,'#')#(Kw  6.-76#V $KWUUVe  skE%N=A1M0 AN=%A N=1AN N=N$AN=0 M? :N= N N= N:5N=9N::N== O r)rFN) rrrrrrCrr5rrrDr+rEr) staticmethodboolrO classmethodrVrYrrr^rntuplerwrr<r7rrrrrr rArA\s 37Hht-./6FJ&A!BCJ)NE(    $ $&&c346s64F6466. B t/0$s); < B B% (.I)J K% c%%NN. |X&;<< =N.N.`MLlMLx8M/NMLMLrrA)"rer'loggingrrrpathlibrtypingrtorch._dynamo.utilsrrrtorch._utils_internalrtorch.utils._filelockr runtime.runtime_utilsr utilsr r getLoggerrrR dataclassrr"r+r2r7r<rArrr rs'  GG1*33! d#  $  d#33$3 d#  $  d#**$* d#00$0 d#99$9xLxLr