Ei@H%SSKJr SSKrSSKrSSKrSSKrSSKrSSKJr SSK J r SSK J r J r SSKJrJr SSKrSSKrSSKJr SSKJr SSKJs Jr SSKJr SSKJr SS KJr S S K J!r!J"r"J#r# S S K$J%r% S S K&J'r'J(r(J)r) \ (aSSK*J+r+J,r,J-r- SSK.J/r/ SSK0J1r1 \"S5r2\"S5r3\Rh"\55r6S&Sjr7\S'Sj5r8\#S(Sj5r9S)S*Sjjr:S+Sjr;\#S,Sj5r<"SS\Rz5r>\#S-Sj5r?\#S.Sj5r@S/SjrA\RRrC\CR\CR\CR\CR\CR\CR\CR\CR\CR\CR\CR\CR\CR\CRR\CRR\CR\CR\CR\CR\CR\CR\CR1r[\"\[5r[\#S,Sj5r\S0Sjr]S1S jr^Sq_S!\`S"'S2S#jraS3S$jrbS4S5S%jjrcg)6) annotationsN)contextmanager)partial)Any TYPE_CHECKING) ParamSpecTypeVar)SymInt)get_decompositions) bind_symbols) aot_function aot_modulemake_boxed_compiler)strip_overloads)default_partition draw_graph#min_cut_rematerialization_partition)Callable GeneratorSequence)Node) IntLikeType_P_RcURRS[RRR S9H,n[RRR UlM. UR5 U$)N call_functionoptarget) graph find_nodestorchopsaten_to_copytor recompile)fx_gnodes W/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/_functorch/compilers.py _canonicalizer,/s[ %% 599>>#:#:&iinn''  NN Kc## [RRS5nSv [RRU5 g![RRU5 f=f7f)NF)r#_C_jit_set_autocast_mode)old_jit_autocast_flags r+_disable_jit_autocastr28sL"HH;;EB?  ''(=>''(=>s A+A A+!A((A+cp[5 [U5 URRS[R R RS9Htn[UR5S:XdM[UR5S:XdM9SUR;dMK[R R RUl Mv URRH]n0nURR5H4upE[U[R 5(a UR"nXSU'M6 X2l M_ URR%5 UR'5 [R(R+U5n[R,R/UR5 [R(R1UR355n[R(R5U5n[7SU55(dU"U6 SSS5 U$!,(df  W$=f)z Compiles the :attr:`fx_g` with Torchscript compiler. .. warning:: This API is experimental and likely to change. Args: fx_g(fx.GraphModule): The input Fx graph module to be compiled. Returns: Torch scripted model. rrr dtypec3j# UH)n[U[RR5v M+ g7fN) isinstancer# _subclasses FakeTensor).0ts r+ ts_compile..ns&M1:a!2!2!=!=>>s13N)r2rr!r"r#r$r%r&lenargskwargsr'r nodesitemsr7devicetypelintr(jitscriptr/_jit_pass_remove_mutationfreezeevaloptimize_for_inferenceany)r)inpsr* new_kwargskvfs r+ ts_compilerRCs  JJ))uyy~~'>'>* D499~"s4;;'71'<DKKAW#iinn//  JJ$$DJ ))+a..A !1 ,%K %   II  T " **1773 II  QVVX & II , ,Q /MMMM tH; !< H= ! < HsA"H&1H& H&E>H&& H5cD[UR5 [XUS9 U$)N) clear_meta)printcoder)r)_namerTs r+_draw_graph_compilerYss $))tj1 Kr-c0[[[US95$)NrX)rrrYr[s r+draw_graph_compiler\{s w':F GGr-cU$)z Returns the :attr:`fx_g` Fx graph module as it is. This is a no-op compiler and can be used to check accuracy. .. warning:: This API is experimental and likely to change. r)rWs r+nopr`s  Kr-cT^\rSrSrSSS.SU4SjjjrS U4SjjrSrU=r$) DebugInterpreterNT initial_envenable_io_processingcZ>[UR/UQ76Ul[TU]"X1US.6$)Nrd)r modulesymbol_mappingsuperrun)selfrerfr? __class__s r+rkDebugInterpreter.runs< + KK   w{ AU  r-c.>^^ ^ ^ ^ SU4Sjjm S U 4Sjjm S U 4Sjjm S U U 4Sjjn[TT]U5nSUR;a[R"URS5upE[R"U5upg[ U5[ U5:wa"[ [ U5S[ U535e[[[ U55XF5H5um p[U [R5(dM'U"XU U4Sj5 M7 U$) Nc >[U[5(dU$[R"URR R TR55nUR(d[SU35e[U5$)Nzexpected r to be a number, got ) r7r sympyexpandr*exprxreplaceri is_numberAssertionErrorint)nirrls r+ subst_symint/DebugInterpreter.run_node..subst_symints_b&))  RWW\\2243F3FGHA;;$'Fqc%JKKq6Mr-c.>[U4SjU55$)Nc34># UH nT"U5v M g7fr6r^)r:rxrzs r+r<HDebugInterpreter.run_node..subst_symint_tuple..s8Cbb))Cs)tuple)nisrzs r+subst_symint_tuple5DebugInterpreter.run_node..subst_symint_tuples8C88 8r-c>T"UR55S:ae[UR5HLnT"URU55URU5:wdM/T"UR U55S:dML g g)Nrr FT)numelrangendimstridesize)abidxrzs r+check_significant_strides.check_significant_stridessbAGGI&* =C$QXXc]3qxx}D(59$ ) r-c z>[U5(d[S[U535eURUR:wa,[U"5SURSUR35eT"UR 55UR 5:waK[U"5SUR 5ST"UR 55SUR 535eT"X5nU(dK[U"5SUR 5ST"UR 55SUR 535eg)Nz"expected desc to be callable, got z:  != z aka )callablervrDr4rr)nvrvdesc same_stridesrrs r+check(DebugInterpreter.run_node..checksD>>$'I$t*%VWWxx288#$xr"((4z%JKK!"''), 9$vhb 51CBGGI1N0OtTVT[T[T]S^_5RSTSTR3$)Nzoutput z where ri)irlsr++DebugInterpreter.run_node..ss'$:M:M9N&Or-)rxrreturnrw)rztuple[IntLikeType, ...]rztuple[int, ...])r torch.Tensorrrrbool)rrrrrzCallable[[], str]rNone) rjrun_nodemetapytree tree_flattenr>rvziprr7r#Tensor)rlnrryn_vals_n_specr_vals_r_specrrrrrzrrms` @@@@r+rDebugInterpreter.run_nodes  9    G Q  AFF?$11!&&-@OF$11!4OF6{c&k)$F }DV %FGG s6{!3VD 2!"ell33bOPEr-r)r?rrezdict[Node, Any] | Nonerfrrr)rrrr)__name__ __module__ __qualname____firstlineno__rkr__static_attributes__ __classcell__)rms@r+rbrbsE/3%)    ,  #    55r-rbc,[U5R$)z Returns a (slow) interpreter over the FX graph module that also checks various debugging properties (e.g., that tracing strides matched real strides.) )rbrkr_s r+ debug_noprs D ! % %%r-c[U5 [RRU5n[RR UR 55nU$r6)rr#rFrGrIrJ)r)rWrQs r+simple_ts_compilers=D A "A Hr-c"[U[5$r6)rr)rQs r+nnc_jitrs , --r-c0[UR5 U$r6)rUrVr_s r+ print_compiler s $)) Kr-c [[[[S.nURU5 [ U[ R R5(a [U40UD6$[U40UD6$)a: Wrapper function over :func:`aot_function` and :func:`aot_module` to perform memory efficient fusion. It uses the :func:`min_cut_rematerialization_partition` partitioner to perform efficient recomputation. It uses NVFuser to compile the generated forward and backward graphs. .. warning:: This API is experimental and likely to change. Args: fn (Union[Callable, nn.Module]): A Python function or a ``nn.Module`` that takes one or more arguments. Must return one or more Tensors. **kwargs: Any other overrides you want to make to the settings Returns: Returns a ``Callable`` or ``nn.Module`` that retains the eager behavior of the original :attr:`fn`, but whose forward and backward graphs have gone through recomputation optimizations, and the graphs have been compiled with nvfuser.  fw_compiler bw_compiler partition_fndecompositions) rRrdefault_decompositionsupdater7r#nnModulerr)fnr@configs r+memory_efficient_fusionrsZ6"!;0 F  MM&"ehhoo&&"'''B)&))r-cURS5 [SUVs/sHo"RUR4PM snS35 SSKJn U"5R 5"U6 [X5$s snf)NfooaQ ############################################################## # To minimize FX graph, copy and paste the below and run it # ############################################################## import torch import torch.fx as fx from functorch.compile import minifier, check_nvfuser_subprocess, check_nvfuser_correctness_subprocess inps = a? inps = [torch.ones(shape, dtype=dtype, device='cuda') for (shape, dtype) in inps] from foo import FxModule mod = FxModule().cuda() with torch.jit.fuser("fuser2"): # check_nvfuser_subprocess can be replaced with check_nvfuser_correctness_subprocess minifier(fx.symbolic_trace(mod), inps, check_nvfuser_subprocess) r)FxModule) to_folderrUshaper4rrcudarR)r)rMrrs r+ debug_compiler9so NN5  &**T''177 T*+, ( JOOt d !! +s!A0 rw graph_indexc P/n[US5n[R"U5n/nUHn[U5S:XaUnU"[R"55nOUupWpn U [ R [ R[ R[ R[ R [ R[ [1;a[ R"SSXyU S9nO[ R"XyU S9nURU5 M SSS5 U$!,(df  U$=f)zR Return a random input for the given inputs meta generated from _save_fx_default. rbr r)r4rCN)openpickleloadr>randomr#rwint32int64ruint8floatrandintrandappend) input_data_pathinputsrQ inputs_metarrDinput_r_strider4rCs r+ get_inputsr[s"$F ot $kk!n D4yA~fmmo.6:3WVIIKKKKJJIIKK  #]]1aFSF"ZZ6JF MM& !'  %. M/ % $. Ms C=D D%c ^^^^ ^ SSKJn S U 4Sjjm S UUUU 4Sjjm S U 4SjjnS U 4SjjnS U 4SjjnU"UUUUU[S9$) a The forward, backward, and joint computation graph will be stored in {folder_name}/{current_name}/{current_name}_forward_{graph_index}, {folder_name}/{current_name}/{current_name}_backward_{graph_index}, and {folder_name}/{current_name}/{current_name}_joint_{graph_index} respectively. The input shape of the graphs will be stored in the .input files. These files can be loaded with pickle, and is a list of format (type, shape, stride, dtype, device). In the case of type = int or float, it is just (type,). For joint graph input, it is a nested list [[],[]] where the two inner lists have the same format. If dump_example_input is True, example_inputs will be stored in .pt file. Since each function might produce multiple graphs, the graph_index is used to distinguish difference graphs r)aot_module_simplifiedc>/n[U5S:a6[US[5(aUT"US5- nUT"US5- nU$UHn[U5[Ld[U5[ LaUR [U545 MDUR [U5URUR5URUR45 M U$)Nrr ) r>r7rrDrwrrrrr4rC)r? input_metaargget_input_metas r+r(_save_fx_default..get_input_metas t9q=ZQ77 .a1 1J .a1 1J CCyC49#5!!49,/!!#Y 3::<CJJO r-c2>[URR5S:Xa,[R[R STU[ 5 g[R"U5nURR[RRR55 UR5 T "U5n[R"TST3SS9 UR!TSTSTSUS[ 3 5 [#TSTSTSUS[ STSUS[ S3S5n[$R&"XE5 SSS5 T(a8[R("UTSTSTSUS[ STSUS[ S 35 gg!,(df  NN=f) Nrz!No nodes in graph {%s}_{%s}_{%s}./T)exist_okrWz.inputwbz.pt)r>r!rAlogloggingWARNINGrcopydeepcopy set_codegenr#fxCodeGenr(osmakedirsrrrdumpsave) gm_to_saver? type_namegmrrQ current_namedump_example_input folder_namers r+graph_saver_helper,_save_fx_default..graph_saver_helpers z%% &! + GG3    ]]: & UXX^^3356 #D)  {m1\N3dC m1\N!L>9+Q{m T m1\N!L>9+Q{mSTUaTbbcdmcnnop{o|}C D   KK &   JJ-qa ~Qyk;-WXYeXffghqgrrsttAADE     s )F Fc>T"XS5 U$)Nforwardr^rexample_inputsrs r+graph_saver_forward-_save_fx_default..graph_saver_forwards 2y9 r-c,>T"XS5 [S- qU$)Nbackwardr )rrs r+graph_saver_backward._save_fx_default..graph_saver_backwards 2z:q  r-c,>T"XS5 [X5$)Njoint)r)r joint_argsrs r+graph_saver_joint+_save_fx_default..graph_saver_joints 273 00r-r)r?rrz list[Any])rfx.GraphModuler?rrstrrr)rrrlist[torch.Tensor]rr)rrrrrz%tuple[fx.GraphModule, fx.GraphModule])functorch.compilerr) rrrrrrrr rrrs ``` @@r+_save_fx_defaultrzs,8 !"!*-!:=! !!F ,>   ,> 1 1(:1 .1 ! '(&-  r-c(Sq[[XU5$)aK Dump the forward, backward, and joint computation graph. Example Usage: save_fx_func = graph_dumper_aot(current_name, folder_name, dump_example_input = False) optimize_ctx = torchdynamo.optimize( save_fx_func ) with torch.enable_grad(): with optimize_ctx: result = forward_and_backward_pass(model, example_inputs) r)rrr)rrrs r+graph_dumper_aotrsK #\@R SSr-)r)rrr)rzGenerator[None, None, None])r)rrMz Sequence[Any]rtorch.jit.ScriptModule)T) r)rrWrrXrrTrrr)rXrrz5Callable[[fx.GraphModule, list[Any]], fx.GraphModule])r)rrWrrr)r)rrWrrzDCallable[[DebugInterpreter, Any, dict[Node, Any] | None, bool], Any])r)rrWrrr)rQCallable[..., Any]rr)rCallable[_P, _R] | nn.Moduler@rrr)r)rrMzSequence[torch.Tensor]rr)rrrr) rrrrrrrztorch.fx.GraphModulerrrz nn.Module)F)rrrrrrrz Callable[[bool, nn.Module], Any])d __future__rrrrrr contextlibr functoolsrtypingrrtyping_extensionsrr rqr#torch.fxrtorch.nnrtorch.utils._pytreeutils_pytreerr torch._decompr %torch.fx.experimental.symbolic_shapesr aot_autogradrrr compile_utilsr partitionersrrrcollections.abcrrr torch.fx.noder torch.typesrrr getLoggerrrr,r2rRrYr\r` Interpreterrbrrrr$r%detach gelu_backwardleaky_relu_backwardsigmoid_backwardthreshold_backwardhardtanh_backwardhardsigmoid_backwardhardswish_backward tanh_backward silu_backward elu_backwardcudnn_batch_normcudnn_batch_norm_backward masked_fillScalarrelu leaky_reluhardtanh hardswish hardsigmoid conj_physical is_same_sizerrrrr__annotations__rrrr^r-r+rFs1"  %%0 $$,>GG*=="'t_ T] ! ??, , `AE  (+9=H H:H   Er~~EP& & &I&&  . yy~~KK""HHOOMMNN-4,,BC $*$$*$*"$*N" " 6""> S>ffff  f ' f  fVEJTT$'T=AT%Tr-