EizSSKrSSKrSSKrSSKrSSKrSSKrSSKJs Js J r SSK Jr SSK J r SSKJr SSKJrJrJrJrJrJr SSKJrJrJrJrJrJrJrJr SSK J!r!J"r" SSK#J$r$ SS K%J&r&J'r'J(r( /S Qr)\r*\RV\RXRV\RZ\RXRZ0\RXRV\RRV\RXRZ\RRZ0S .r.S r/S!S jr0S"Sjr1Sr2Sr3S#Sjr4S$Sjr5Sr6Sr7\Rp"\&5S%Sj5r9Sr:Sr;\Rp"\&5S&Sj5r<\Rp"\&5S\RzSS4Sj5r>\Rp"\&5S&Sj5r?\Rp"\&5S#Sj5r@\Rp"\&5S'Sj5rAS(SjrBS#SjrCS)S jrDg)*N) _FusedModule)_is_activation_post_process)_activation_is_memoryless_add_module_to_qconfig_obs_ctrdefault_dynamic_qconfigfloat16_dynamic_qconfig!float_qparams_weight_only_qconfig&float_qparams_weight_only_qconfig_4bit)_get_special_act_post_process_has_special_act_post_process)get_default_dynamic_quant_module_mappingsget_default_qat_module_mappings$get_default_qconfig_propagation_list(get_default_static_quant_module_mappings2get_default_static_quant_reference_module_mappingsno_observer_set) DeQuantStub QuantWrapper)type_before_parametrizations)DEPRECATION_WARNINGget_qparam_dict)has_no_children_ignoring_parametrizations) get_default_custom_config_dictpropagate_qconfig_add_quant_dequantpreparequantizequantize_dynamic prepare_qat quantize_qatconvert swap_module)%float_to_observed_custom_module_class)observed_to_quantized_custom_module_classc[$)z'Defines the default custom config dict.)_DEFAULT_CUSTOM_CONFIG_DICT[/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/ao/quantization/quantize.pyrrGs &&r)cUR[U5U5nURX55n[USU5n[RR R RXP5 [XP5nX`lUR5H]upxU(aUS-U-OUn Ub8XtRS/5;aM0[U5URS/5;aMQ[XXi5 M_ g)aThis is a helper function for `propagate_qconfig_` Args: module: input module qconfig_dict: dictionary that maps from name of submodule to quantization configuration qconfig_parent: quantization config of parent module, we will fallback to this config when there is no specified config for current module prefix: corresponding prefix of the current module, used as key in qconfig_dict prepare_custom_config_dict: dictionary for custom handling of modules see docs for :func:`~torch.ao.quantization.prepare_fx` Return: None, module is modified inplace with qconfig attached qconfig.Nnon_traceable_module_namenon_traceable_module_class) getrgetattrtorchao quantizationr,_assert_valid_qconfigrnamed_childrentype_propagate_qconfig_helper) module qconfig_dictqconfig_parentprefixprepare_custom_config_dictmodule_qconfigqconfig_with_device_checknamechild module_prefixs r*r8r8Ls2"%%$V,nN"%%f=NVY?N HH!!77O >~ V.N,,. /5 t+4 % - 223NPRS SE{)--.JBOP &%> /r)c,Uc0nUc0n[XUS9 g)acPropagate qconfig through the module hierarchy and assign `qconfig` attribute on each leaf module Args: module: input module qconfig_dict: dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute) prepare_custom_config_dict: dictionary for custom handling of modules see docs for :func:`~torch.ao.quantization.prepare_fx` Return: None, module is modified inplace with qconfig attached N)r=)r8)r9r:r=s r*rr}s)  !)%'"9Sr)c$URU5$)z.Forward hook that calls observer on the outputactivation_post_process)selfinputoutputs r*_observer_forward_hookrJs  ' ' //r)c*URUS5$)z2Forward pre hook that calls observer on the outputrrE)rGrHs r*_observer_forward_pre_hookrLs  ' 'a 11r)Fc[US5(d [S5eU(aUR[SS9 gUR [ SS9 g)NrFzGExpect activation_post_process attribute already attached to the moduleT)prepend)hasattrAssertionErrorregister_forward_pre_hookrLregister_forward_hookrJ)r9pre_hooks r*&_register_activation_post_process_hookrTsM 64 5 5 U  (()CT(R$$%;T$Jr)c^^ ^ Uc [5nUc0nTcM[U5n[U5S:a[SU35e[U5S:a[ [ U55OSmS Sjm Sm S UU U 4SjjnUR 5GHupx[U5[RLaM$[[U5[R[R45(aRT "U5(aC[US5(d[S [U5S 35eT "URT5UlMM[#U[$5(aT "U5(a U"U5 MMUb*[U5U;aT "U5(a U"U5 GMGM ['U5(a[)U5n U"X5 GM/T "U5(aj[U5U;a[U[U5n U R+U5n [-XU 5 [U [/[1555(d U"U 5 GMGM[3UUUTU5 GM [5U5(a@[#U[6RR85(d[U5U;aU"U5 [US 5(aC[#U[6RR85(d[U5U;a U"U5 gggg) aGAdd observer for the leaf child of the module. This function insert observer module to all leaf child module that has a valid qconfig attribute. Args: module: input module with qconfig attributes for all the leaf modules that we want to quantize qconfig_propagation_list: a list of quantizable modules that will have observers added to them if they are leaf nodes device: parent device, if any non_leaf_module_list: list of non-leaf modules we want to add observer Return: None, module is modified inplace with added observer modules and forward_hooks NrzR_add_observer_ only works with cpu or single-device CUDA modules, but got devices rcbUcUR5OU"5nUbURU5 U$N) activationto)r,devicespecial_act_post_processrXs r*get_activation_post_process3_add_observer_..get_activation_post_processs=(/    )+    MM& !r)cD[US5=(a URSL$)Nr,rOr,)ms r*needs_observation)_add_observer_..needs_observationsq)$>$)>>r)c>T"U5(aX[U[5(dBURST"URTU55 [ U[ UR5S9 ggg)z]Adds an activation post process module and register a pre or post hook that calls the module rFrSN) isinstancer add_moduler,rTr)r`r[rZr\ras r*insert_activation_post_process6_add_observer_..insert_activation_post_processsa Q   1k(B(B LL)+IIv'?  35aii@ )C r)rFzfunctional class z- has no pre-defined `activation_post_process`weight_fake_quantrW)r_get_unique_devices_lenrPnextiterr6rnnDropout issubclassnnqFloatFunctional QFunctionalrOr,rFrerr r from_floatsetattrtupler_add_observer_rr2 Sequential)r9qconfig_propagation_listnon_leaf_module_listrZcustom_module_class_mappingdevicesrgr@rAr[observed_classobserved_childr\ras ` @@r*rwrws, '#G#I "*&(#~&v. wH ($+  U/j 2&996588#6#677 ( 04L L&v. +,,6588#6#677 ( 04L L&v. M8 -r)c0UR5Vs1sH*oRRS:wdMURiM, snUR5Vs1sH*oRRS:wdMURiM, sn-$s snfs snf)Nmeta) parametersrZr7buffers)r9ps r*rjrj6sy$//1 M1XX]]f5LHAHH1 M ..*Q*Qhhmmv.E*Q  MQsBBB9Bc[U5(a-[US5(aUR(a [U5$UR 5Hup[ U5UR U'M U$)aOWrap the leaf child module in QuantWrapper if it has a valid qconfig Note that this function will modify the children of module inplace and it can return a new module which wraps the input module as well. Args: module: input module with qconfig attributes for all the leaf modules that we want to quantize Return: Either the inplace modified module with submodules wrapped in `QuantWrapper` based on qconfig or a new `QuantWrapper` module which wraps the input module, the latter case only happens when the input module is a leaf module and we want to quantize it. r,)rrOr,rr6r_modules)r9r@rAs r*rr<s[ 2&99 FI & & NNF##,,.  1% 8/ Mr)cz[RRS5 Uc [5nUR S05nU(d[ R "U5nUnUc [5n[USS9 [SUR555(d[R"SSS9 [UUUUS 9 U$) aPrepares a copy of the model for quantization calibration or quantization-aware training. Quantization configuration should be assigned preemptively to individual submodules in `.qconfig` attribute. The model will be attached with observer or fake quant modules, and qconfig will be propagated. Args: `model`: input model to be modified in-place `inplace`: carry out model transformations in-place, the original module is mutated `allow_list`: list of quantizable modules `observer_non_leaf_module_list`: list of non-leaf modules we want to add observer `prepare_custom_config_dict`: customization configuration dictionary for prepare function .. code-block:: python # Example of prepare_custom_config_dict: prepare_custom_config_dict = { # user will manually define the corresponding observed # module class which has a from_float class method that converts # float custom module to observed custom module "float_to_observed_custom_module_class": {CustomModule: ObservedCustomModule} } z!quantization_api.quantize.prepareNr$r:c3`# UH$n[US5=(a URv M& g7f)r,Nr_).0r`s r* prepare..s#LOqwq)$22Os,.zNone of the submodule got qconfig applied. Make sure you passed correct configuration through `qconfig_dict` or by assigning the `.qconfig` attribute directly on submodules) stacklevel)r{)r2_C_log_api_usage_oncerr0copydeepcopyrranymoduleswarningswarnrw)modelinplace allow_listobserver_non_leaf_module_listr=r{rys r*rrWsD HH  !DE!)%C%E""<"@"@/#  e$ *#G#I u40 LEMMOL L L  K    %$?  Lr)c^[TS5(a&[TR5(a [TS5 SU4SjjnU"SS9 U"SS9 g)NrFFc>U(a TRO TRnU(a[O[n[ 5nUR 5HupEXRLdM UR U5 M UHnURU5 M grW)_forward_pre_hooks_forward_hooksrLrJsetitemsaddpop)rShook_map observer_hookhandle_ids_to_remove handle_idhook_fnr9s r* remove_hooks5_remove_activation_post_process..remove_hookssp086,,f>S>S*2 &8N  #u"*.."2 I'$((3#3.I LL #.r)TrdF)rOrrFdelattr)r9rs` r*_remove_activation_post_processrsMv0116Q&&77 12 $$% r)cUR5Hn[U5 M [US5(aU?[ U5 g)zzClean up the qconfig left in the module so that new qconfig can be propagated. Args: module: module to be cleaned up r,N)children_remove_qconfigrOr,r)r9rAs r*rrs9"#vy!! N#F+r)c[RRS5 Uc [5nU(d[R "U5nUR 5 [USS9 U"U/UQ76 [XSS9 U$)aSQuantize the input float model with post training static quantization. First it will prepare the model for calibration, then it calls `run_fn` which will run the calibration step, after that we will convert the model to a quantized model. Args: model: input float model run_fn: a calibration function for calibrating the prepared model run_args: positional arguments for `run_fn` inplace: carry out model transformations in-place, the original module is mutated mapping: correspondence between original module types and quantized counterparts Return: Quantized model. z"quantization_api.quantize.quantizeTr) r2rrrrrevalrr")rrun_fnrun_argsmappingrs r*rrsd$ HH  !EF:<  e$ JJL E4  58 ED) Lr)c [RRS5 UGcU[R:Xa|[R [ [R[ [R[ [R[ [R[ [R[ 0nGOU[R:Xa|[R [[R[[R[[R[[R[[R[0nGO4U[R:Xa+[R[ [R"[ 0nOU[R$:Xa[R[&0nO[)SUS35e[+U[,5(aU[RLa[ nOcU[RLa[nOIU[RLa[ nO/U[R$La[&nO[/S[1U55e[3[5U[6R8"U555nUc [;5nU(d[<R>"U5nURA5 [CX5 [EXSS9 U$)a*Converts a float model to dynamic (i.e. weights-only) quantized model. Replaces specified modules with dynamic weight-only quantized versions and output the quantized model. For simplest usage provide `dtype` argument that can be float16 or qint8. Weight-only quantization by default is performed for layers with large weights size - i.e. Linear and RNN variants. Fine grained control is possible with `qconfig` and `mapping` that act similarly to `quantize()`. If `qconfig` is provided, the `dtype` argument is ignored. Args: model: input model qconfig_spec: Either: - A dictionary that maps from name or type of submodule to quantization configuration, qconfig applies to all submodules of a given module unless qconfig for the submodules are specified (when the submodule already has qconfig attribute). Entries in the dictionary need to be QConfig instances. - A set of types and/or submodule names to apply dynamic quantization to, in which case the `dtype` argument is used to specify the bit-width inplace: carry out model transformations in-place, the original module is mutated mapping: maps type of a submodule to a type of corresponding dynamically quantized version with which the submodule needs to be replaced z*quantization_api.quantize.quantize_dynamicz5Don't know how to quantize with default settings for z. Provide full qconfig pleasez.Unknown dtype specified for quantize_dynamic: Tr)#r2rrqint8rnLinearrLSTMGRULSTMCellRNNCellGRUCellfloat16rquint8 EmbeddingBagr Embeddingquint4x2r ValueErrorrer RuntimeErrorstrdictzip itertoolsrepeatr rrrrr")r qconfig_specdtyperrdefault_qconfigs r*rrs@ HH  !MN EKK  20/ 4 3 3 Lemm # 20/ 4 3 3 Lell "!B ?Lenn $!GLGwNkl  L# & & EKK 5O emm #5O ell "?O enn $DO@#e* C i.>.>.OPQ ;=  e$ JJLu+ ED) Lr)c>[RRS5 UR(d [ S5eUc [ 5nU(d[ R"U5n[USS9 [XSSS9 [U[UR55SS9 U$) a Prepares a copy of the model for quantization calibration or quantization-aware training and converts it to quantized version. Quantization configuration should be assigned preemptively to individual submodules in `.qconfig` attribute. Args: model: input model to be modified in-place mapping: dictionary that maps float modules to quantized modules to be replaced. inplace: carry out model transformations in-place, the original module is mutated z%quantization_api.quantize.prepare_qatz1prepare_qat only works on models in training modeNrTF)rrremove_qconfig)rr) r2rrtrainingrPrrrrr"rrvalues)rrrs r*r r >s|  HH  !HI >>PQQ13  e$u40 EDG EW^^5E1FPTU Lr)c[RRS5 U(d[R"U5nUR 5 [ USS9 U"U/UQ76 [USS9 U$)aCDo quantization aware training and output a quantized model Args: model: input model run_fn: a function for evaluating the prepared model, can be a function that simply runs the prepared model or a training loop run_args: positional arguments for `run_fn` Return: Quantized model. z&quantization_api.quantize.quantize_qatTr)r2rrrrtrainr r")rrrrs r*r!r!]sW HH  !IJ  e$ KKMt$ 58 E4 Lr)c [RRS5 U(d[R"U5n[ UUSUUUS9 U(a [ U5 U$)adConverts submodules in input module to a different module according to `mapping` by calling `from_float` method on the target module class. And remove qconfig at the end if remove_qconfig is set to True. Args: `module`: prepared and calibrated module `mapping`: a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules `inplace`: carry out model transformations in-place, the original module is mutated `convert_custom_config_dict`: custom configuration dictionary for convert function `use_precomputed_fake_quant`: a flag to enable use of precomputed fake quant .. code-block:: python # Example of convert_custom_config_dict: convert_custom_config_dict = { # user will manually define the corresponding quantized # module class which has a from_observed class method that converts # observed custom module to quantized custom module "observed_to_quantized_custom_module_class": { ObservedCustomModule: QuantizedCustomModule } } z!quantization_api.quantize.convertT)r is_referenceconvert_custom_config_dictuse_precomputed_fake_quant)r2rrrr_convertr)r9rrrrrrs r*r"r"usUJ HH  !DE v& !#=#=  Mr)c UcU(a [5O [5nUc [5nURS05nU(d[R "U5n0nUR 5HEup[U [5(d[U 5U;a[U USUUUS9 [XXe5Xx'MG UR5HupXRU 'M U$)aCConverts submodules in input module to a different module according to `mapping` by calling `from_float` method on the target module class Args: module: input module mapping: a dictionary that maps from source module type to target module type, can be overwritten to allow swapping user defined Modules inplace: carry out model transformations in-place, the original module is mutated is_reference: a flag to enable quantized reference module use_precomputed_fake_quant: a flag to enable use of precomputed fake quant r%Tr)rrrr0rrr6rerrrr#rr) r9rrrrrr{reassignr@modkeyvalues r*rrs, ? @9;  ")%C%E""<"@"@3R# v&H**, 3 --,S19TT *+E  % 5 -&nn& $' Mr)cUn[US5(Ga6URGb(Sn[U5U;a U[U5RU5nSnO[U5U;aU[U5n[US5(arUR(aaURc [ S5eURR 5nU"UR 5 [U5nURX5nOQ[R"UR5n SU R;aURXS9nOURU5nSnU(aURR5Hn URU 5 M URR5Hn U [ LdMUR#U 5 M! [%U5n ['U 5S ::d7['U 5S :Xa[(R*"S 5U ;d[ S U 35e['U 5S :a[-[/U 55OSn U (aUR1U 5 U$)zSwaps the module if it has a quantized counterpart and it has an `observer` attached. Args: mod: input module mapping: a dictionary that maps from nn module to nnq module Return: The corresponding quantized module of `mod` r,NFT _IS_REFERENCEzAmodule qconfig must not be None when swapping to reference modulerrrrrzOswap_module only works with cpu or single-device CUDA modules, but got devices r)rOr,r from_observedrrPweightrrtinspect signaturerrrrQrrJrRrjrkr2rZrlrmrY)rrr{rnew_modswappedqmodweight_post_processweight_qparamssig pre_hook_fnrr|rZs r*r#r#sGsI3;;#: ' ,0K K1,S1mC  G )# .' 97<=Dt_--$2D2D;;&(['*kk&8&8&:##CJJ/!01D!E//#>''8/3>>A"oo.G#ooc2GG "55<<> 11+> ?--446"8811':7 +3/GG !LA%%,,v*>'*I$efmeno-0L1,.get_prefix4s2v76C<7r)rFN)rOrFr6_get_observer_dict)r target_dictr<rr@rArBs r*rr+si8s-..  ' ' Jv&)BBC))+ 5; 6*T1 5}=,r))NrN)NNr)NNNN)FNNN)NF)NFTFNF)NFFNF)r)Errrtyping_extensionsrr2torch.ao.nn.quantizedr3rn quantizedrqtorch.nntorch.ao.nn.intrinsicrtorch.ao.quantization.observerrtorch.ao.quantization.qconfigrrrrr r +torch.ao.quantization.quantization_mappingsr r r rrrrrtorch.ao.quantization.stubsrrtorch.nn.utils.parametrizerutilsrrr__all__is_activation_post_processr quantizableMultiheadAttentionr'rr8rrJrLrTrwrjr deprecatedrrrrrrr r!r"rr#rr(r)r*rsE  ##.F   BC 9 $$ r~~@@. R\\.. ))2<<+J+J2 ' # .b20 2 K"  $ H/V 612 "&# @3@F!4 , 123:12EKKuW3Wt123<123.12  #$131l  #$ ;~KP@F>r)