Eis SrSSKrSSKrSSKJr SSKJr SSKJrJ r SSK J r SSK J r SSKrSSKJr SS KJr SS KJr \ "S \\\4\\\\\44-\ -5r\ r\ "S \\\\4-\\\\\44-\ -5r"S S5r\R4R6\R4R8\R4R:\R4R<\R4R>\R4R@\R4RB\R4RD\R4RF\R4RH\R4RJ\R4RL\R4RN\R4RP\R4RR1r*\R4RVRX\R4RVRZ\R4RVR\\R4RVR^\R4RVR`\R4RVRb\R4RVRd\R4RVRf\R4RVRh\R4RVRj\R4RVRl\R4RVRn\R4RVRp\R4RVRr\R4RVRt\R4RVRv\R4RVRx\R4RVRz\R4RVR|\R~\R\R|\R\R\R\R\R\R1rG\RSSSSSSSSSSSSSSSSSS S!S"S#S$S%S&1rIS'rJS(rKS)rLS*rMS+\ S,\NS-\ 4S.jrOS/rPS0rQS1rRS2rSS3rTS4rUS5rVS6rWS7rXS8rYS9rZS-\[4S:jr\S;r]S<r^S=\RS>\RS-\[4S?jr`S@\aSA\aSB\[SC\RSD\[S-\\a\a44 SEjrcSFrdSGreSH\R4RSI\R4RS-\NS-4SJjrgSK\SL\h\N\ 4S-\h\N\ 44SMjriSK\S-SN4SOjrjSP\SL\h\N\ 4S-SN4SQjrkS@\aSA\aS-S4SRjrl\R4S=\RS>\RS@\aSA\aSC\RSS\RSB\[ST\RS-\\R\R44SUjjroSK\S-\a4SVjrpSW\R4RSX\\ SY4S-\h\N\\ SY444SZjrqS[\R4RS-\ 4S\jrrS]rs/S^Qrtg)_z? Utils shared by different modes of quantization (eager/graph) N) OrderedDict)Callable)getfullargspec signature)Any) TypeAliasType) QuantType)Node)is_parametrized NodePatternPatternc\rSrSrSrSrg) MatchAllNode/zfA node pattern that matches all nodes, used in defining fusion patterns in FX Graph Mode Quantization N)__name__ __module__ __qualname____firstlineno____doc____static_attributes__rX/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/ao/quantization/utils.pyrr/srrrelurelu_ contiguousdetachdetach_ hardsigmoid hardsigmoid_permuterepeatrepeat_interleavereshaperesize_shapesigmoidsigmoid_sizesqueezesqueeze_tanhtanh_ transpose unsqueeze unsqueeze_viewc*URS:H=(a UR[;nURS:H=(a UR[;nURS:H=(a( [ U[ UR55[ ;nX#U4$)N call_function call_method call_module)optarget func_list method_listtypestrmodule_type_list)nodemodulesis_call_functionis_call_methodis_call_modules r check_noderBswww/1NdkkY6NWW -L$++2LN = XT'#dkk2B*C%DHX%X ^ ;;rcHUR5nURU5 U$)a{ Combines two dictionaries. This function takes two dictionaries as input and returns a new dictionary that contains all the key-value pairs from both input dictionaries. If there are any duplicate keys in the `additional_dict`, the values from the `additional_dict` will overwrite those in the `default_dict`. Args: default_dict (dict): The main dictionary that will be used as the base additional_dict (dict): The dictionary used to update `default_dict` Returns: dict: The resulting dictionary Example: >>> x = dict(a=1, b=1) >>> y = dict(b=2, c=3) >>> get_combined_dict(x, y) {'a': 1, 'b': 2, 'c': 3} )copyupdate) default_dictadditional_dictds rget_combined_dictrIs#( AHH_ Hrc\U[R:H=(d U[R:H$N)torchper_tensor_affineper_tensor_symmetricqschemes r is_per_tensorrQs# e-- - VEG " ( (E!2G j!7MM%00W))   e11 1..G0883G9=>>!I.@@BEG&L%{335?? %{335??  Nrc[U5nURU05n[U5U;a[S[U5SU35eU[U5$)aGet the observed/quantized custom module class that we need to swap `custom_module` to Input: custom_module: input, can be an instance of either a float or observed custom module custom_module_class_mapping: the float to observed or observed to quantized custom module class mapping qconfig: qconfig configured for the custom module Output: corresponding observed/quantized custom module class for input custom module instance z5did not find corresponding observed module class for z in mapping: )get_quant_typegetr:rn) custom_modulecustom_module_class_mappingqconfig quant_type class_mappings rget_swapped_custom_module_classrsh (J/33JCM M-/ $] 34M- R   m, --rcVUc [S5eUR5nUR$)Nz6qconfig must be provided to determine activation dtype)rn activationrt)rrs ractivation_dtypers-UVV##%J   rcVUc [S5eUR5nUR$)Nz2qconfig must be provided to determine weight dtype)rnweightrt)rrs r weight_dtypers)QRR ^^ F <<rc x[U5[R[R[R[R [R [R[R[R[R[R4 ;=(a [U5(+$)zGiven a qconfig, decide if the activation needs to be quantized or not, this includes quantizing to quint8, qint8 and qint32 and float16 ) rrLrbrdrffloat16rcrerkrgrlrm#activation_is_dynamically_quantizedrs r"activation_is_statically_quantizedrsw G $           )  =37; ; =rc"[U5upnU$)zGiven a qconfig, decide if the activation needs to be dynamically quantized or not, this includes dynamically quantizing to quint8, qint8 and float16 )get_qconfig_dtypes)r_activation_dtype_activation_is_dynamics rrr's 3EW2M//  rc[U5[R[R[R[R 4;$)zzGiven a qconfig, decide if the activation needs to be quantized to int8 or not, this includes quantizing to quint8, qint8 )rrLrbrdrcrers ractivation_is_int8_quantizedr0s7 G $    ) rcZ[U5[R[R4;$)zPGiven a qconfig, decide if the activation needs to be quantized to int32 or not )rrLrfrgrs ractivation_is_int32_quantizedr<s! G $u{{(C CCrc J[U5[R[R[R[R [R [R[R[R[R[R4 ;$)zCGiven a qconfig, decide if the weight needs to be quantized or not ) rrLrbrdrrhrcrerkrgrlrmrs rweight_is_quantizedrCse             %  rc[U5[R[R[R[R 4;$)zNGiven a qconfig, decide if the weight needs to be statically quantized or not )rrLrbrdrcrers rweight_is_statically_quantizedrUs-  U\\5;; UZZ$X XXrc[U5upnU[R[R4;=(a, U[R[R 4;=(a U$)zLGiven a qconfig, returns True if this op is using int8 dynamic quantization )rrLrbrcrdre)rrrrs r op_is_int8_dynamically_quantizedr\sV=OwU[R[R4;aSU s=:aS::d O [ S5eU(a US -US -pX4$U[R[R4;aU(aS upX4$S upX4$U[R [R4;aU(aS upX4$SupX4$U[R[R4;aS upX4$U[R:XaSupX4$U[R:XaSupX4$SupX4$)zlCalculates actual qmin and qmax based on the quantization range, observer datatype and if range is reduced. )rl)rrzRquantization range should be positive and not exceed the maximum bit range (=256).lzYquantization range should be positive and not exceed the maximum bit range (=4294967296).r)i?)i)rr)ii)ri)ii)r) rLrfrgrdrernrbrcrjrk) rxryrrtrinitial_quant_mininitial_quant_maxcustom_quant_mincustom_quant_max qrange_lens rcalculate_qmin_qmaxrs U\\5;;/ /3? 0 039 0 .7*  ',<,H   1 ':Q> U[[%**- - )c)$h*u||U[[1 1 +e+$o #,>9>y*  % U[[%**- -'.$  (1$ 9  u||U[[1 1'-$   (.$ 9  u||U[[1 1#< I   ell "#/ I   ekk !#6 I  $) I  rcfURSS5n[U5S:XaSUS4$USUS4$)z$ Turn 'foo.bar' into ['foo', 'bar'] r[rr)rsplitlen)r7rs r _parent_namers>  c1A 1v{1Q4xtQqTzrc[UR5S:Xag[U5(a.[UR5S:H=(a SUR;$g)z Checks if module._modules is empty or if module is a parametrization, checks that module._modules only has the 'parametrizations' module rTrparametrizationsF)r_modulesr )modules r)has_no_children_ignoring_parametrizationsrsI  6??q   6??#q(R-?6??-RRrroot submodulecFUR5H up#XLdM Us $ g)a.Get the path (fully qualified name) of a submodule Example:: >> class M(torch.nn.Module): def __init__(self) -> None: self.linear = torch.nn.Linear(5, 5) def forward(self, x): return self.linear(x) >> m = M() >> l = m.linear >> _get_path_of_module(m, l) "linear" N) named_modules)rrnps r_get_path_of_modulers($""$ >H% rfloccUR5VVs0sH"up#U[U5R;dM X#_M$ snn$s snnf)zGet local keyword arguments Example:: >> def f(self, a, b=9): pass >> loc = {"a": 6, "c": 7} >> _get_signature_locals(f, loc) {"a": 6} )itemsr parameters)rrkvs r_get_signature_localsr+s9!YY[ I[TQA11H1H,HDAD[ II Is AAzOrderedDict[str, Any]cB0n[U5RR5Hlup#URURLaURX'M.UR UR LaSX'MMUR URLdMh0X'Mn [U5$)zGet all default keyword arguments from function signature Example:: >> def f(self, a, b=9): pass >> _get_default_kwargs(f) {"b": 9} r) rrrdefaultemptykindVAR_POSITIONAL VAR_KEYWORDr)rkwargsnameparams r_get_default_kwargsr9sF |..446  == + ==FL ZZ5// /FL ZZ5,, ,FL 7 v rfuncc[U5n[X5nUR5nUR5HupVXT;dM XdU'M U$)aGiven a function and local function arguments, normalize the keyword arguments by filling in default arguments from function signature Example:: >> def f(self, key1=3, key2=3): pass >> loc = {"key2": 6} >> _normalize_kwargs(f, loc) {"key1": 3, "key2": 6} )rrrDr)rrdefault_kwargs local_kwargsnormalized_kwargsattrvals r_normalize_kwargsrNsR).N(3L&++-!'')   $&)d #* rcXUSs=::aU::d O [S5eX:a [S5eg)a=Validates that the user-specified quantization range is properly initialized and within the given bound supported by the observer dtype. To accommodate lower-bit quantization with respect to the existing torch.qint8 and torch.quint8 datatypes, the user can choose to use dynamic quantization range by passing in a tuple of initial qmin and qmax values. One use case is these customized qmin and qmax values are used to calculate static estimates of the scale and zero point for aggressive lower-bit fake quantization. These estimates are compared against parameters learned through backpropagation. The related literatures for scale and zero point via backpropagation are as follows: Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf rz1Used-specified quantization range must include 0.zKqmin must be strictly less than qmax for user-specified quantization range.N)rn)rxrys rvalidate_qmin_qmaxrds8  'i 'PQQ Y  repsrPc[X5(dT[R"S/URRS9[R"S/URRS94$[R "U[R "U55n[R"U[R "U55n URn [R"UR5[RU S9n [R"UR5[RU S9n URU 5nU[R:XdU[R:Xa[R"U*U 5n U [!X2- 5S- - n [R"X5n U[R"[R$4;aMU(a&U R'U R5X#-S-5n OU R'U R5S5n OU[R(:XaIX- [!X2- 5- n [R*"X:U [R,"U 55n SU-U - n OwX- [!X2- 5- n [R"X5n U[R."X- 5R[R05- n [R2"XU5n [5U R65S:Xa*[R"[!U 5/U R8U S9n [5U R65S:Xah[R"[1U 5/U R8U S9n U[R(:Xa*[R"[!U 5/U R8U S9n U R[R5U R[R54$)aDCalculates the quantization parameters, given min and max value tensors. Works for both per tensor and per channel cases Args: min_val: Minimum values per channel max_val: Maximum values per channel Returns: scales: Scales tensor of shape (#channels,) zero_points: Zero points tensor of shape (#channels,) g?)devicer)rtrr)rrLtensorrr:min zeros_likemaxonesr)doublezerosint64torNrUrrcrbnew_fullrTwhere ones_likeroundintclamprr&rt) rrrxryrtrrrP min_val_neg max_val_posrrvrws rdetermine_qparamsrs* w 0 0||SE'..*=*=> C++A   ))GU%5%5g%>?K))GU%5%5g%>?K   F JJ{'')f ME[--/u{{6RJ &&.C%,,,5;V;V0Vii k: uY%:;a?@ %% U[[%,,/ /$'00OO% (=!'C (001BCH E:: :"eI,A&BB EK0FG '\E) *eI4I.JJ %%[-@!A!D!DUYY!OO [[ B  5;;1 eEl^5;;vN :  !\\ _ Z%5%5f  e<< <z"#:+;+;FJ 88ELL !:==#= ==rc>[[U5R5$)zGet number of positional args for a function Example:: >> def f(self, key1=3, key2=3): pass >> _get_num_pos_args(f) 3 )rrargs)rs r_get_num_pos_argsrs ~a %% &&rmodelexample_inputs.c<^^^Um0mUUU4Sjn[RRRmU[RRlU"U6 T[RRlT$!T[RRlf=f)aGiven a model and its example inputs, return a dictionary from fully qualified name of submodules to example_inputs for that submodule, e.g. {"linear1": (tensor1,), "linear2": (tensor2,), "sub": (tensor3,), "sub.linear1": (tensor4,), ...} Used to make quantizing submodules easier now that FX Graph Mode Quantization requires example inputs. Also works for keyword arguments with default values, we would flatten keyword arguments as positional arguments and fill in the missing keyword args with default values, e.g. if we have a forward function: def forward(self, x, key1=3, key2=3): ... and we call it with self.submodule(x, key2=6) we'll get example_inputs: (x, 3, 6) user can also override `key1` with positional arguments as well: for self.submodule(x, 5, key2=6) we'll get: (x, 5, 6) variable positional arguments and variable positional keyword arguments in forward function are not supported currently, so please make sure no submodules is using them. c>[U5R5n[URU5n[ UR5S- nU[ U5- nU(a+U(a$UR SS9 US-nU(a U(aM$URUR55 [U5n[T U5nUbUT U'T "U/UQ70UD6$)NrF)last) listrDrforwardrrpopitemextendvaluestupler) selfrrsubmodule_example_inputsrnum_args num_to_popsubmodule_example_inputs_tuplerXfqn_to_example_inputsorig_module_callrs r_patched_module_call7get_fqn_to_example_inputs.._patched_module_calls#':??#4 -dllFC$T\\2Q6$< == .  % %5 % 1 !OJ.. !''(9(@(@(BC)./G)H&!$- ?)G !# &6t6v66r)rLnnModule__call__)rrr'r%r&rs @@@rget_fqn_to_example_inputsr,st8 D7 xx//3EHHOO4 ~$4  $4 s A::!BrcUR5Vs1sHoRiM snUR5Vs1sHoRiM sn-n[R"S5[R"S51U:Xa,[R "SSS9 [R"S51n[ U5S:a[SU35e[ U5S:a[[U55nU$S nU$s snfs snf) zz Returns the unique device for a module, or None if no device is found. Throws an error if multiple devices are detected. cpumetazfBoth 'meta' and 'cpu' are present in the list of devices. Module can have one device. We Select 'cpu'.rrrzKprepare only works with cpu or single-device CUDA modules, but got devices rN) rrbuffersrLrrrrnnextiter)rrdevicesrs r_assert_and_get_unique_devicer4s "(!2!2!45!4Axx!45 ..*9*Q*9G  UU\\&12g= t <<&' 7|a &i )  %(L1$4T$w- F M;?F M'69s C:C?atorch.ao.quantization is deprecated and will be removed in 2.10. For migrations of users: 1. Eager mode quantization (torch.ao.quantization.quantize, torch.ao.quantization.quantize_dynamic), please migrate to use torchao eager mode quantize_ API instead 2. FX graph mode quantization (torch.ao.quantization.quantize_fx.prepare_fx,torch.ao.quantization.quantize_fx.convert_fx, please migrate to use torchao pt2e quantization API instead (prepare_pt2e, convert_pt2e) 3. pt2e quantization has been migrated to torchao (https://github.com/pytorch/ao/tree/main/torchao/quantization/pt2e) see https://github.com/pytorch/ao/issues/2259 for more details)r r rrBrIrQrVr`rrrrrrrrrrrrrrrrr,rqrrDEPRECATION_WARNING)urr\r collectionsrcollections.abcrinspectrrtypingrtyping_extensionsrrL torch.ao.quantization.quant_typer torch.fxr torch.nn.utils.parametrizer rr QuantizerClsr rr)ReLUReLU6AdaptiveAvgPool1dAdaptiveAvgPool2dAdaptiveAvgPool3d AvgPool1d AvgPool2d AvgPool3d MaxPool1d MaxPool2d MaxPool3dIdentity HardsigmoidSigmoidTanhr< functionaladaptive_avg_pool1dadaptive_avg_pool2dadaptive_avg_pool3delu hardswish instance_norm layer_norm leaky_relusilumishdropout max_pool1d max_pool2d max_pool3drhardtanh hardtanh_rr'r.r#r*stacksumr,r/catr8meanr9rBrIrQrVr;r`rqrrrrrrrrrrboolrrrTensorrr rtrrrr*rdictrrrrrMrPrrr,r4r5__all__rrrrgsO#$-+ 665t$uT5t3D-D'EEK      Hh   HeHh./ /01    HHMM HHNN HH HH HH HH HH HH HH HH HH HH HH HH HHMM$ HH++ HH++ HH++ HH HH!! HH%% HH"" HH"" HH HH HH HH"" HH"" HH"" HH HH   HH!! HH## HH OO  MM MM KK II JJ OO II9  > JJ               3 << 2W:#:C:C:!(!H..=$! D$Y   <"J >= = =  =  ;; =  =  38_ = @  ((//&+hhoo4Z0 JX JDcN JtCH~ J8(?*H4S>>U, # # $ H#44K> \\K> \\K>K> K> ;; K>  K> K>]]K> 5<< %&K>\ ' 'c '6! 88??6!,1#s(O6! #uS#X 6!r%((//c8 E r