EirSSKrSSKJr SSKJr SSKJrJrJr SSK r SSK r SSK J r SSK Jr SSKJr SS KJr SS KJrJrJrJrJrJrJr /S Qr\R8"\5r\ "S S 9"SS\55r \ "S S 9"SS\55r!\ "S S 9"SS\55r"\"SS55r#"SS5r$g)N)Callable) dataclass)AnycastOptional) compatibility)map_arg) ShapeProp) split_by_tags)CALLABLE_NODE_OPSFxNetAccFusionsFinderNamesNodeListNodeSetTensorOrTensorsTensors)FxNetMinimizerBadModuleErrorFxNetMinimizerRunFuncError!FxNetMinimizerResultMismatchErrorF)is_backward_compatiblec\rSrSrSrSrg)r"z1 Raised if failed to split out a minimize module N__name__ __module__ __qualname____firstlineno____doc____static_attributes__rY/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/fx/passes/net_min_base.pyrr"r"rc\rSrSrSrSrg)r)z8 Raised if error occurs during run_a or run_b functions rNrrr"r#rr)r$r"rc\rSrSrSrSrg)r0zB Raised if comparing function thinks the results are mismatching. rNrrr"r#rr0r$r"rcf\rSrSr%SrSr\\S'Sr\ \S'Sr \\S'Sr \\S'Sr \\S 'S r S rg ) _MinimizerSettingBase7a Args: `accumulate_error`: Instead of using a's input for both converted module to verify , use the previous outputs of each converted module as input to accumulate the errors. `traverse_method`: "sequential" or "binary" or "accumulate" Determine the way of traverse the nodes in FX module. `find_all`: Minimizer will go through the entire model and return all problematic nodes. `return_intermediate`: If true, when using `run_nodes()` function to run the model, intermediate results of all the ops will be returned as output. `all_outputs`: If true, when using `_run_and_compare()` function, all the output nodes in the subgraph will be used for comparison. Faccumulate_error sequentialtraverse_methodfind_allreturn_intermediate all_outputscfSn[U5R5Hup#USUSUS3- nM U$)NzFX Minimizer Settings:  :  )varsitems)self settings_strkvs r#__str___MinimizerSettingBase.__str__Qs@1 J$$&DA b2aSO +L'r"rN)rrrrr r,bool__annotations__r.strr/r0r1r<r!rr"r#r*r*7sB$#d"'OS'Hd %%Kr"r*c\rSrSrSrS8S\R RS\S\ \ \ \ /\ \ \44S\S\\ \\R R\/S4S \\ \\\/S44 S jjrS9S jrS:S \R RS\S\S \ 4SjjrS:S \R RS\S\S \ 4SjjrS\ S\ S\R R4SjrS\R RS\S \ \\44SjrS\4SjrS\S \ \R R\44SjrS:S\R RS\S\ S\4S jjrS!\S"\S#\S \4S$jrS\S \4S%jrS\S \4S&jr S\S"\S#\S'\S \\4 S(jr!S\S'\\S \4S)jr"S\S \4S*jr#S\S \4S+jr$S!\S"\S#\S \4S,jr%S!\S-\&S \4S.jr'S/\\S0\\S \4S1jr(S8S/\\S0\\4S2jjr)S3\&\4S4jr*S5r+S;S/\\S0\\S-\\&S'\\S \4 S6jjr,S7r-g)<_MinimizerBaseZa This class is used to automatically find problematic nodes in a model. It takes a FX graphmodule and generate some submodules while traverse the graph. Then two functions `run_a` and `run_b` will be used to run the same submodule and a function `compare_fn` will be used to compare the results. Currently we provides two ways to traverse the graph and generate submodules. 1. Sequential traversal: this will traverse the graph node by node and generate one submodule with one single node. 2. Binary searching: this will do a binary search style traversal on the graph. For internal Users, a guide can be found here https://fb.quip.com/HDtuAgiKGfkP. Nmodule sample_input compare_fnsettingsmodule_exporter exclusion_fnc[U[RR5(d[ S[ U535eXlX lX0lXPl X@l X`l 0Ul 0Ul 0Ul/UlSUlUR R"R$Vs1sHowR&[(;dMUiM nnUR+5 [-UR U5"5UlUR R"R$Vs/sH owR&S:XdMUR0PM" n n[3U 5[3UR5:wa.[ S[3U 5S[3UR5S35e[5U 5H'upX*URU 'X*URU 'M) gs snfs snf)NzExpected GraphModule, got r placeholderzPlaceholder count (z%) does not match sample_input count ()) isinstancetorchfx GraphModuleAssertionErrortyperDrErFrHrGrI a_outputs b_outputsresultsreports iterationgraphnodesopr run_shape_proprfusionsnamelen enumerate) r8rDrErFrGrHrInodecallable_nodes placeholdersir]s r#__init___MinimizerBase.__init__is&%(("6"677 #=d6l^!LM M ($. (*,*,(* )+  "[[..44 4TCT8TD4   ,T[[.IK #'++"3"3"9"9 "9$WW =UIDII"9   | D$5$5 6 6 %c,&7%89''*4+<+<'=&>aA  !.GA#/?DNN4 #/?DNN4 /#  s*GGG6Greturnc\[UR5R"UR6 g)zy Helper function to run shape propagation on module. Can be overridden by subclasses for custom shape propagation logic. N)r rD propagaterE)r8s r#r[_MinimizerBase.run_shape_props! $++(($*;*;0_MinimizerBase._store_outputs..s 2Tgg6IDD2s" "rN) nextrXrYrMargsrNrONoderSr]rTr_)r8rwrxry output_nodercargs r#_store_outputs_MinimizerBase._store_outputss  &__22  k&&q)588== 9 97?NN;++A.33 47?NN;++A.33 4$K$4$4Q$78+3;sxx(+3;sxx(9r" main_module submod_pathc^ /m /n[X5nURRVs/sH oURS:XdMURPM" nn[ U5UR R5::aFUH?nT RUR U5 URURU5 MA OURR(a[SUS35 S[RRS[ 4U 4SjjnUR#U5n U"UR$6 U R'5 T nURR(dT T 4$T U4$s snf)a Try get submodule inputs from stored outputs. If not found then use torch_glow.get_submod_inputs to get the inputs. If accumulate_error is False, use a_input for run_a() and run_b() otherwise use a_input for run_a and b_input for run_b. Args: main_module: Top-levlel fx module. submod_path: Path to the submodule we want to run and compare results. Returns: a_input: List of tensor(s) that will be used by run_a() as submodule inputs. b_input: List of tensor(s) that will be used by run_b() as submodule inputs. rKz)Can't find previous stored outputs named !r8rkc >UmgNr)r8rka_inputs r# get_inputs5_MinimizerBase._get_submod_inputs..get_inputss r")getattrrXrYrZr]setrSkeysappendrTrGr,printrNnnModulerregister_forward_pre_hookrEremove) r8rrb_inputryr`rbr]rhandlers @r#_get_submod_inputs!_MinimizerBase._get_submod_inputss1$K5 "+//"7"7 "7$77m;SIDII"7   |  3 3 5 5$t~~d34t~~d34%}}--A,qQR ! !# ! 88DF ** + MMOG}}--G# #; s E&E&selected_nodescURRRHXnUR[;aMX!;a SUlM'[ SUR55(a SUlMQSUlMZ g)a/ Tag selected nodes with tag "minimize". Nodes with the same tags will be split to the same submodule afterwards. Args: selected_nodes: Nodes that we want to minimize. We will tag those nodes with "minimize", all preceding nodes with "main_0" and all following nodes with "main_1". minimizec3l# UH*nUR[;dMURS;v M, g7f)>main_1rN)rZr tag)r}ns r#r~,_MinimizerBase._tag_nodes..s0-A44,,0//-s44rmain_0N)rDrXrYrZr ranyall_input_nodes)r8rr`s r# _tag_nodes_MinimizerBase._tag_nodesskKK%%++Dww//%%-- $#,r"rYcURU5 [UR/SQ5nSnUR5H#upESU;aM US:XaUnM[ SU35e US:Xa[ SU35eX#4$)a3 Split self.module so that one submodule consists of `nodes` and only `nodes`. Args: nodes: Nodes that we want to include in the minimize submodule. Returns: split_module (torch.fx.GraphModule): the module after split. submodule_name (str): the name of the submodule that consists of `nodes`. )rrrrz0Expected only one minimize submodule with nodes z,Minimize submodule was not found with nodes )rr rDnamed_childrenr)r8rY split_modulesubmodule_name child_name_s r#_build_submodule_MinimizerBase._build_submodule(s %T[[2RS !)88:MJ+#!+2FugN; R .>ugF ++r"r submod_name output_namescp[X5nURX5upg[UR5S:Xa"URR /5 SUlURUS:aUOUR S- nUR S5 U(aUR R(d/n URRHQn U RS:XaURRU 5 U RU;dM@U R U 5 MS URR[U 5S:XaU SO [U 55 URR5 UR!5 URRH,n U RS:XdM[#U R$S5n M. UR'XVU5n UR)XWU5n UR+XU5 UnUcW Vs/sHn[1U5PM nnUR3XU5unnUUR4W 'UR S U35 U(dUR S U 35 UR6(ai[9U [5(aU S n [9U [5(a[1U S5n UR7UUU S -5 UR7UUU S -5 [;S U 35eg![,a-nUR SUSU35 [/SUSU35eSnAff=fs snf)ae Run the submodule in `split_module` that has name `submod_name` using `self.run_a` and `self.run_b` and compare their results. Args: split_module: Main module that contains the minimize submodule. submod_name: Name of the minimize submodule. output_names: Names of the node we want to output. If None, we will use the original output. rr zRun and compare ...r|cUR$r)r])xs r#1_MinimizerBase._run_and_compare..{s!&&r"zException raised when running r4NzNumerical accuracy = zResult mismatch for _cpu_acc)rrr^rVrrWrGr1rXrYrZ erase_noder]r|tuplelint recompiler rrqrur Exceptionrr@rFrUrHrMr)r8rrrrlryrrreport output_nodesr` result_keyrwrxenamesr;numeric_result bool_results r#_run_and_compare_MinimizerBase._run_and_compareNs"L6 22<M t||  ! LL   #DNJ!OjRSAST +,  9 9%'L!--77h&OO..t499 , ''- . OO " "#&|#4#9 Qu\?R  OO "    !OO))Dww("$TYY0@A * zz)jAHzz)jAH   I >$  %/0ZSVZE0&*ooh%&P# #1 Z  -n-=>? MM0 = >##j%00!+BJj%00!$Z]!3J$$' $$'  46J:,4WX X3  MM:;-r!M N,0 RsC  1s6K9L39 L0(L++L0 all_nodes start_idxend_idxc[5nXUn/nURbAURXRU5 [U5S:Xa S/nURR U5 U$USR nUSR nU=R S- slURR U5 UR SUR 35 UR SUSUSUS- SUS [U53 5 [U5n URU 5upURXU/5 UR S 5 URU5 [5$![[4a [U5S:Xa*UR S U S 35 URU5 U s$UR S 5 URU5 [U5S-n URXX,-5n[U5S:waURR(dUs$URXU -U5n[U5S:Xa%UR SU35 URU5 Us$f=f)z) Recursive binary search implementation. rAll nodes are excluded by userrr zBinary search iteration From node index : to &. Size of the interested node list is No discrepancy found.zcThis is the last node in the sub-module. Search in the current branch is successful with culprit = .MProceed to split and lower the halves of the current sub-module individually.zaFurther split and lowering found no errors. Unable to minimize the submodule with list of nodes: )rrIr^rVrr]rWrr print_reportrr_binary_search_implrGr/) r8rrrculpritsrYrfirst_node_nameoutput_node_name cur_nodesrrmids r#r"_MinimizerBase._binary_search_impls^  E#g6    (   e 85zQ:; ##F+(-- 9>> ! F# 00@AB yk?*;4! }AN^M_`336u:, @ !Z & (,(=(=i(H %L  ! !,>N=O PD MM1 2   f %5LE+,MN 5zQ QQZP[[\^!!&)  MM+    f %e*/C// ioVH8}!$--*@*@// s?GTH8}! LLQ7T!!&)O= s!&EAIA0IA IIc:URUS[U55$)z' Binary search on `nodes` for culprit. r)rr^)r8rYs r#_binary_traverse_MinimizerBase._binary_traverses''q#e*==r"c[5nUGHyn/nURRU5 U=RS- slURSURS35 URSUR35 [ R SUR5 U/nURbsURUSS5 [U5S:XaQURSUR35 URU5 URR(dUs $GMU1nX0R;aURUnURU5upxURXxUR/5 URU5 GM| U$![a\ UR!U5 URS U35 URU5 URR(dUss $GM["a\ UR%U5 URS U35 URU5 URR(dUss $GMIf=f) zH Traverse `nodes` one by one and determine if any of them is a culprit. r zSequential traverse iteration rz Visit node: zVisit node: %srrzUser exclusion : "Found culprit from numeric error: Found culprit from run error: )rrVrrWr]_LOGGERinforIr^rrGr/r\rrraddrupdate) r8rYrr`r node_listrrrs r#_sequential_traverse#_MinimizerBase._sequential_traverses ED "F LL   ' NNa N MM:4>>:J!L M MML 4 5 LL)499 5#'&I  ,!!)R4y>Q&MM$5dii["AB%%f-==11' "&I||# LL.  $,0,A,A),L) %%l$))M!!&)7R5 $ T" B4&IJ!!&)}}--#O.- $ * >tfEF!!&)}}--#O.  $sAF  AI2AIIfind_last_nodec  /nX#-S-nU(aUSUS-OXSnUR(aURUSS5 [U5nUSRn USRn U(aU OU n U=RS- slURR U5 UR SSUR3/5 UR SU(aS OS S 3S US XRSUS XRS3 SU SU 3SU 3SX2- S-3/5 [UR5S- n URU5upURXU /U 5 UR SUS XRS35 X&s=:XaU:XahO OeUS:XaU(aU[U5S- :Xa7U(a0UR SU(aS OS S35 URU5 gX$(aS-$S-$UR S5 URU5 U(aURXS-X45$URXXd5$![[4a UR SU SU S35 X&s=:XaU:Xa>O O;UR SSUSXRS3/5 URU5 Us$UR S5 URU5 U(aURXXd5s$URXS-X45s$f=f)z Recursive block search implementation. find_last_node: If True, search for the last node which result in numerics difference if False: find first node in sorted node list rNr rrz==============================zBlock search iteration z Search for lastfirstz node in culpritsrrrz. zSubgraph constructed by zTargeting node: z$Size of the interested node list is z&Culprits not found from node start to rzAt z node, no culprits found.rzCulprits found from node z)This is the last node in the sub-module. z6Search in the current branch is successful with node :z , node name: )rIrr]rWrVrextendr^rrr_block_traverse_implrr)r8rYrrrrrcur_nodes_listrrlast_node_nametarget_node_namerlrrs r#r#_MinimizerBase._block_traverse_impl&s_"q(7E537#35QU;      nb" 5' (+00'+00-;> ! F# )$..)9:   fGDDUV"9+Qu/?/D/D.ET'RSTYTbTgTgShhjk*?*;4?OP"#3"456w7JQ7N6OP   &* ? X(,(=(=i(H %L  ! !N+;Z @ MM8Quz>OqQ *7*N>Ua/NMMfGDD]^%%f-!A@@R@@ MM+    f %007G003WWu23MN  MM+O+ threshold 2nd pass: search for start_idx by finding the first node in culprit block where Numerical accuracy (start_idx, end_idx) < threshold Form minimum block by (start_idx - 1, end_idx) rrzBlock search from rz2**************************************************r Nz(Start searching for last node in culpritTzNo culprits foundz Finish Pass 1zFind end_idx = rz)Start searching for first node in culpritFz Finish Pass 2zFind start_idx = z'Finish searching, found minimum block (,rL) rr]rrVr^rrrrintr)r8rYrrrrlast_node_reportrrfinal_start_idx final_end_idxrun_bothfirst_node_report range_start range_end result_reports r#_block_traverse_MinimizerBase._block_traversesq E(--r00AnEUVW) ,- e*q.)2'. !T) ~  # #$N O   . / 55eQUVM$ ''(;<!!"23  # ##%m_Ae6J6O6O5PQ    . / >!L M    / 0"77a'A+& 3KGUO& ''(;<!!"23  $ $X . LL   1 2  $ $#''8%:P:U:U9VW    / 0"&c?!;T#}=UYIM:;5e6H5I5K[J\\] ^  M* -(r"c[5nURbURUSS5 [U5S:Xa S/nURR U5 U$USR nUSR nSUSU3/n[U5nUR U5upxURXxU/5 URU5 U$![[4a* UR SU35 URU5 Us$f=f)z< run user defined `nodes` and determine if it is a culprit. rrrzDefined graph from rFound culprit ) rrIr^rVrr]rrrrr) r8rYrrrrrrrs r#_defined_traverse _MinimizerBase._defined_traverses  E    (   eR , u:?67F LL   'O(-- 9>>''8=MN=O P   f %  23MN  MMN9+6 7   f %O s 7C7D?Dc.[5n[5nURR(a [S5 U$UGHn/nURR U5 U=R S- slUR SUR S35 URU5 URnUb[U[5(aUSnUb[U[5(d[SU35eUR SU35 URU5upxURXxU/5 URU5 GM U$![ ["4a= URU5 UR SU35 URU5 Uss $f=f) Nz9'Find All' mode is not supported in accumulate traversal.r zAccumulate traverse iteration rrzminimize: node_name: z Add node: r)rrGr/rrVrrWrr]rMrr@rQrrrrr) r8rYr nodes_to_runr`r node_namerrs r#_accumulate_traverse#_MinimizerBase._accumulate_traversesiE #  == ! ! M NOD "F LL   ' NNa N MM:4>>:J!L M   T " I$Iu)E)E%aL  9c(B(B$'ykJ K   f %O)  OOI & MM:9+F G   f %O  s5%EAG AG G skip_nodescSn[U5nSn[5nXT:a_XnURU;aXS:aURXU5nUS-nO"XTS- :XaX5::aURXUS-5nUS- nXT:aM_U$)rrr )r^rr]r)r8rrr num_nodesidxrr`s r#_skip_traverse_MinimizerBase._skip_traverseKs  N 5o>DyyJ&?#77 cRH!G A %)*:33I#PQ'R 1HCor"startendc/nUSLnURRRHVnUR[;aMUR U:XaSnU(aUR U5 UR U:XdMU U$ U$)zr Collect nodes in the model that between nodes with name of `start` and `end`. These two nodes are also included. NT)rDrXrYrZr r]r)r8rrrYadd_noder`s r#_collect_nodes_MinimizerBase._collect_nodes_sw D=KK%%++Dww//yyE! T"yyC , r"cURX5n[U5nUH2nXPR;dMURURU5 M4 /nURR (aUVs/sHoUR PM nnURU5upxURXxU5 gs snf![[4an [U 5 Sn A gSn A ff=f)a Run part of the model from `start` node to `end` node. If `start` is None then we start from the beginning of the model. If `end` is None then we stop at the end of the model. Args: start: The name of the node which is the first node of the submodule we want to run. If set to None, then we'll start with the first node of the model. end: The name of the node which is the last node of the submodule we want to run. If set to None, we'll end with the last node of the model. N) rrr\rrGr0r]rrrrr) r8rrrYrr`rrrrs r# run_nodes_MinimizerBase.run_nodesvs##E/J D||#  d!34 == , ,278%$II%L8 (,(=(=i(H %L  ! !,\ J 9 ' -   !HH  s6B4%B99C CCrc[[U55H(nUS:a[SX-5 M[X5 M* g)Nrz . )ranger^r)r8rrcs r#r_MinimizerBase.print_reports5s6{#A1uefi'(fi $r"cLURHnURU5 M gr)rVr)r8rs r# print_reports_MinimizerBase.print_reportssllF   f %#r"c[UR5 [URR5 UR X5nURR S:XaUR U5$URR S:XaURU5$URR S:XaURU5$URR S:XaUc [S5eURXS5$URR S:XaURU5$URR S:XaURXT5$[SURR S 35e) a Minimizing the model from node with name `start` to node with name `end` base on self.settings. Find culprits that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors. Args: start: The name of the node where we want to start minimizing. If set to None, then we'll start with the first node of the model. end: The name of the node where we want to terminate minimizing. If set to None, we'll end with the last node of the model. skip_nodes: The names of nodes where we want to skip during minimizing. It'll create subgraphs without these skip nodes under the hood. Only applicable in mode "skip". find_last_node: True if only last_node of a culprits is needed in mode "block". False if only the first_node of a culprits is needed. Only applicable in mode "block". Returns: nodes: A list of nodes that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors during minimizing. r-binary accumulateskipz<'skip_nodes' can't be None when 'traverse_method' is 'skip'.definedblockzUnknown traverse method r) rrGrDrXrr.rrr rorrr)r8rrrrrYs r#r_MinimizerBase.minimizes=: dmm dkk ##E/ == ( (L 8,,U3 3 == ( (H 4((/ / == ( (L 8,,U3 3 == ( (F 2!"R&&u9 9 == ( (I 5))%0 0 == ( (G 3''> >5dmm6S6S5TTUVWWr") rSrTrFrIr\rWrDrHrVrUrErG)NN)rfN)r)NNNN).rrrrr rNrOrPrrrrrfloatr>r*rr@rrrdr[rqrurrrrrrrrrrrrr rlistrrrrr$rr!rr"r#rBrBZs 0 GK93$$9393 ou 5uUD[7I I 93(93" guxx33S94? @ 93x3(A2 w 2 h$$2$,g$,%8L8Lc8Q2R$,V [Yhh**[Y[Y [Y  [YzB!B.1Bh>7> /(/w/bhXhX*-hX8;hXMQhX #hXTMM/7~M M^xG6$($w$L)!).1)r;s$!&& 1!!&    H %e,9- e,- e, -   DAXAXr"