qyi3SSKJr SSKJr \(a SSKJr SSKJr SSKJ r J r J r J r J r SSKJr \ "5(a SS KrSS KJr \ R&"\5rS r"S S \5rg ) ) TYPE_CHECKING) HfQuantizer)PreTrainedModel) Mxfp4Config)is_accelerate_availableis_kernels_availableis_torch_availableis_triton_availablelogging)get_module_from_nameN)WeightConverterc^\rSrSr%SrSrS\S'U4SjrSrSr S S S \ S \ 4S jr SSjr SS S S\ 4SjjrSrSrSrSr\S \ 4Sj5rSrSrSrU=r$)Mxfp4HfQuantizer*z' FP4 quantization using fbgemm kernels Frquantization_configc 6>[TU]"U40UD6 SUlgN)super__init__triton_kernels_hub)selfrkwargs __class__s d/home/mande/repo/quber/.venv/lib/python3.13/site-packages/transformers/quantizers/quantizer_mxfp4.pyrMxfp4HfQuantizer.__init__2s ,77"&cURc SSKJn U"S5UlUR$UR$![a [S5ef=f)z3Lazy import and initialize kernels only when neededr) get_kernelz(kernels-community/gpt-oss-triton-kernelsz2kernels package is required for MXFP4 quantization)rintegrations.hub_kernelsr ImportError)rr s r_lazy_import_kernels%Mxfp4HfQuantizer._lazy_import_kernels6s]  " " * XA*45_*`'&&&t&&& X!"VWW Xs :Ac[5(d [S5eURR(ag[ 5(d [S5e[ R R5=(d [ R"S5nURS;aKUR(a+[RSUS35 SURlg[SUS 35e[ RR5(aSn[!S 5n[#5nO[ R$R5(a9[ R$R'5nUS :n[!S 5n[#5nO.URS:XaSn[!S 5n[#5nOS nS nS nUR(aU(d'[RS5 SURlgU(d'[RS5 SURlgU(d'[RS5 SURlgO6U(d [)S5eU(d [)S5eU(d [)S5eUR(dUR+5 UR-S5nUbH[/U[05(a2UR(d SUR35;a [)S5egggg)NzqUsing mxfp4 quantization requires torchPlease install the latest version of torch ( pip install --upgrade torch )z9Using mxfp4 requires Accelerate: `pip install accelerate`cpu)cudaxpur&zGUsing MXFP4 quantized models requires model on cuda/xpu/cpu, but found zj, we will default to dequantizing the model to bf16. To use mxfp4, please disable the current accelerator.TzIQuantizing a model using MXFP4 requires model on cuda/xpu/cpu, but found z7. To use mxfp4, please disable the current accelerator.z3.5.0)z3.4.0FuMXFP4 quantization is only supported on GPUs with compute capability >= 7.5 (e.g T4, A100, L4, H100, or B200) or XPUs (e.g Intel® Data Center GPU Max Series). We will default to dequantizing the model to bf16.zMXFP4 quantization requires Triton: CUDA requires Triton >= 3.4.0, XPU/CPU requires Triton >= 3.5.0. Please install triton: `pip install triton`. We will default to dequantizing the model to bf16.zMXFP4 quantization requires the `kernels` package: `pip install kernels>=0.12.0`. We will default to dequantizing the model to bf16.uMXFP4 quantization is only supported on GPUs with compute capability >= 7.5 (e.g T4, A100, L4, H100, or B200) or XPUs (e.g Intel® Data Center GPU Max Series) or CPUzMXFP4 quantization requires Triton: CUDA requires Triton >= 3.4.0, XPU/CPU requires Triton >= 3.5.0. Please install triton: `pip install triton`zPMXFP4 quantization requires the `kernels` package: `pip install kernels>=0.12.0` device_mapdiskzYou are attempting to load an FP4 model with a device_map that contains a disk device.This is not supported when the model is quantized on the fly. Please use a quantized checkpoint or remove the disk device from the device_map.)r r"r dequantizer torch acceleratorcurrent_acceleratordevicetype pre_quantizedlogger warning_once RuntimeErrorr( is_availabler r r'get_device_capability ValueErrorr#get isinstancedictvalues) rargsrr1is_device_supported_mxfp4triton_availablekernels_installedcompute_capabilityr+s rvalidate_environment%Mxfp4HfQuantizer.validate_environmentAs!##]   # # . . &((YZ Z""668OELL)#^^,=r2rJF#^^,=BPRSF)66>>EETT$1199>>)B#7=dV1TF'236rs\!0734   H %Q {Q r