qyi>8&SSKrSSKJr SSKJr SSKJrJrJrJ r J r J r J r J r JrJrJrJrJrJrJrJrJrJrJrJrJrJrJrJr SSKJr SSK J!r! SS K"J#r# SS K$J%r% SS K&J'r' SS K(J)r) SS K*J+r+ SSK,J-r- SSK.J/r/ SSK0J1r1 SSK2J3r3 SSK4J5r5 SSK6J7r7 SSK8J9r9 SSK:J;r; SSKJ?r? SSK@JArA SSKBJCrC SSKDJErE SSKFJGrG SSKHJIrI SSKJJKrK SSKLJMrM 0S\%_S \)_S!\+_S"\9_S#\!_S$\C_S%\E_S&\5_S'\7_S(\/_S)\;_S*\=_S+\-_S,\1_S-\K_S.\'_S/\M_\I\3\#\A\?\GS0.ErN0S\_S \ _S!\ _S(\ _S"\_S#\_S$\_S%\_S&\_S'\_S*\_S+\ _S,\ _S)\_S-\_S.\ _S/\_\\\\\\S0.ErO\\\\ \ \\\4rP\R"\R5rS"S1S25rT"S3S45rUS5\V4S6jrWS7\V4S8jrXS9rYg):N) AutoConfig)logging) AqlmConfigAutoRoundConfig AwqConfigBitNetQuantConfigBitsAndBytesConfigCompressedTensorsConfig EetqConfigFbgemmFp8ConfigFineGrainedFP8ConfigFourOverSixConfig FPQuantConfig GPTQConfig HiggsConfig HqqConfig MetalConfig Mxfp4ConfigQuantizationConfigMixinQuantizationMethod QuantoConfig QuarkConfig SinqConfig SpQRConfig TorchAoConfig VptqConfig) HfQuantizer)AqlmHfQuantizer)AutoRoundQuantizer) AwqQuantizer)BitNetHfQuantizer)Bnb4BitHfQuantizer)Bnb8BitHfQuantizer)CompressedTensorsHfQuantizer)EetqHfQuantizer)FbgemmFp8HfQuantizer)FineGrainedFP8HfQuantizer)FourOverSixHfQuantizer)FPQuantHfQuantizer)GptqHfQuantizer)HiggsHfQuantizer)HqqHfQuantizer)MetalHfQuantizer)Mxfp4HfQuantizer)QuantoHfQuantizer)QuarkHfQuantizer)SinqHfQuantizer)SpQRHfQuantizer)TorchAoHfQuantizer)VptqHfQuantizerawqbitsandbytes_4bitbitsandbytes_8bitgptqaqlmquantoquark fouroversixfp_quanteetqhiggshqqzcompressed-tensors fbgemm_fp8torchaobitnetvptq)spqrfp8z auto-roundmxfp4metalsinqc@\rSrSrSr\S\4Sj5r\S5rSr g)AutoQuantizationConfigz The Auto-HF quantization config class that takes care of automatically dispatching to the correct quantization config given a quantization config stored in a dictionary. quantization_config_dictc URS5nURSS5(dURSS5(a/URSS5(aSOSn[RU-nOUc [S5eU[;a,[SUS [ [ R5535e[UnURU5$) N quant_method load_in_8bitF load_in_4bit_4bit_8bitThe model's quantization config from the arguments has no `quant_method` attribute. Make sure that the model has been correctly quantizedUnknown quantization type, got  - supported types are: ) getrBITS_AND_BYTES ValueError AUTO_QUANTIZATION_CONFIG_MAPPINGlistAUTO_QUANTIZER_MAPPINGkeys from_dict)clsrOrQsuffix target_clss Y/home/mande/repo/quber/.venv/lib/python3.13/site-packages/transformers/quantizers/auto.pyr` AutoQuantizationConfig.from_dicts/33NC # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<>>( # #rfrMc~\rSrSrSr\S\\-4Sj5r\S5r \S\\-S\S-4Sj5r \ S 5r S r g) AutoHfQuantizerz The Auto-HF quantizer class that takes care of automatically instantiating to the correct `HfQuantizer` given the `QuantizationConfig`. rhc [U[5(a[RU5nURnU[ R :Xa<[U[5(d [S5eUR(aUS- nOUS- nU[;a,[SUS[[R5535e[UnU"U40UD6$)NzZFound `quant_method=bitsandbytes` but `quantization_config` is not a `BitsAndBytesConfig`.rUrTrWrX) isinstancerwrMr`rQrrZr TypeErrorrRr^r[r]r_)rarhrnrQrcs rd from_configAutoHfQuantizer.from_configs )4 0 0"8"B"BCV"W *77  -<< <13EFFp#//' ' 5 51,@/44678:  ,L9 -888rfc R[R"U40UD6nURU5$)N)rMrjr)rarmrnrhs rdrjAutoHfQuantizer.from_pretraineds*4DDEbmflm233rfquantization_config_from_argsNcJUbSnOSn[U[5(aA[U[5(a[R"U5nO[RU5nUbhUR R UR R :wa:[SUR R SUR R S35e[U[5(aq[U[5(a\UR5nUR5HupV[XU5 M U(a US[UR55S3- nUS:wa8[U[[[ 45(d["R$"U5 U$[&R)U5 U$)zt handles situations where both quantization_config from args and quantization_config from model config are present. zYou passed `quantization_config` or equivalent parameters to `from_pretrained` but the model you're loading already has a `quantization_config` attribute. The `quantization_config` from the model will be used.zThe model is quantized with z but you are passing a z| config. Please make sure to pass the same quantization config class to `from_pretrained` with different loading attributes.z"However, loading attributes (e.g. z]) will be overwritten with the one you passed to `from_pretrained`. The rest will be ignored.)r}rwrr`rM __class__rqr[LOADING_ATTRIBUTES_CONFIG_TYPESget_loading_attributesitemssetattrr]r_rrrwarningswarnloggerinfo)rarhr warning_msgloading_attr_dictattrvals rdmerge_quantization_configs*AutoHfQuantizer.merge_quantization_configss ) 4y  K )4 0 07II&5&?&?@S&T#&<&F&FGZ&[# * 5#--66:W:a:a:j:jj./B/L/L/U/U.VVmoLoVoVo_o_n``FF  )+J K KPZ )+JQ Q !> T T V .446 +377!!CDIZI_I_IaDbCcdA BB " Z0CkS^`tEu%v%v MM+ &#" KK $""rfc URSS5nURSS5(dURSS5(a/URSS5(aSOSn[RU-nOUc [S5eU[;a8[ R SUS [[R55S 35 gg ) NrQrRFrSrTrUrVrWrXz~. Hence, we will skip the quantization. To remove the warning, you can delete the quantization_config attribute in config.jsonT) rYrrZr[r\rwarningr]r^r_)rOrQrbs rdsupports_quant_method%AutoHfQuantizer.supports_quant_method s/33NDI # ' ' > >BZB^B^_motBuBu 8 < <^U S SWY`F-<T[;a[STS35e[U[5(d [ S5eU[T'U$)NzConfig '' already registeredz*Config must extend QuantizationConfigMixin)r\r[ issubclassrr~)rars rdregister_config_fn8register_quantization_config..register_config_fn$sK 5 5xx/CDE E#677HI I36(0 rfri)rrs` rdregister_quantization_configr!s rfnamec^U4SjnU$)zRegister a custom quantizer.c>T[;a[STS35e[U[5(d [ S5eU[T'U$)Nz Quantizer 'rz!Quantizer must extend HfQuantizer)r^r[rrr~)rars rdregister_quantizer_fn1register_quantizer..register_quantizer_fn4sJ ) ){4&0DEF F#{++?@ @'*t$ rfri)rrs` rdregister_quantizerr1s ! rfcH[US5nU(a&[RUR5(dSnU(dUbRU(a&[R URU5UlOXl[R URUS9nOSnUbUR UUS9 URU5nURU5nURU5n[URSS5(d&URRn[USU5US'X`U4$)NrhF) pre_quantized) device_map weights_only dequantizevaluequant) hasattrrzrrhrrvalidate_environmentupdate_device_mapupdate_tp_planupdate_ep_planrkrQ)configrhrr user_agentr hf_quantizerrQs rdget_hf_quantizerrAsF$9:M_BB6C]C]^^ +7 )8)S)S**,?*F &*= &&22  & &'3  ))!% * "33J? ,,V4,,V4|77uMM';;HHL"),"NJw   ++rf)Zrmodels.auto.configuration_autorutilsrutils.quantization_configrrrr r r r r rrrrrrrrrrrrrrrrbaserquantizer_aqlmr quantizer_auto_roundr! quantizer_awqr"quantizer_bitnetr#quantizer_bnb_4bitr$quantizer_bnb_8bitr%quantizer_compressed_tensorsr&quantizer_eetqr'quantizer_fbgemm_fp8r(quantizer_finegrained_fp8r)quantizer_fouroversixr*quantizer_fp_quantr+quantizer_gptqr,quantizer_higgsr- quantizer_hqqr.quantizer_metalr/quantizer_mxfp4r0quantizer_quantor1quantizer_quarkr2quantizer_sinqr3quantizer_spqrr4quantizer_torchaor5quantizer_vptqr6r^r\r get_loggerrqrrMrzstrrrrrirfrdrs74+4'/22F+6@92+-)--/-++1+ <++ O   O     )" O  >6&!  !" O#$  $$   /4$ 9$+$+$ J $  J $  J $ l$ [$$$ $ 9$1$/$ [$}$  !$" J#$$  !   /$ 6  #   H %&#&#Rll^   !S ! !,rf