qyi(8SSKJr SrSrSr"SS\5rg))PushToHubMixinc Sn[UR55nURR nUS:Xa.SSKJn U"5(aSSKn[US5Rbg U$![a [SU35ef=f) zw Infers the device type from the model parameters. Args: model: The model instance. Returns: The device type. z { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } zPCannot determine model device, please provide a device to the mapping. Example: cudar)is_torch_availableNversionrocm) next parameters StopIteration ValueErrordevicetypeutilsrtorchgetattrhip)modelEXAMPLE_MAPPINGparamdev_typerrs ]/home/mande/repo/quber/.venv/lib/python3.13/site-packages/transformers/utils/kernel_config.py infer_devicers O U%%'( ||  H6.    ui(,,8 O!  ^_n^o p   s A))BcSSKJn US;a[SU35eURS5SnURS5SnUX5"UUS900X@'g)NrLayerRepositoryrr xpunpuneuron=Only cuda, rocm, xpu, npu and neuron devices supported, got: :)repo_id layer_name)kernelsrr split)r%r repo_namemodecompatible_mappingrrepo_layer_namer$s radd_to_mappingr,:sl' ==XY_X`abbooc*1-Oooc"1%G /* &"cSSKJn SSKJn US;a[ SU35eUR S5SnUR S5SnUR S5S n UX6"U"U5U US 900X@'g) Nr)Path)LocalLayerRepositoryrr!r"r#/) repo_path package_namer%)pathlibr/r&r0r r') r%rr(r)r*r/r0r+r3repo_package_names radd_to_mapping_localr7Ks, ==XY_X`abbooc*1-O$Q'I!,R0 &y/.* &"r-cB\rSrSrSrS SjrS SjrSrSrS Sjr S r g) KernelConfig`z] Kernel configuration class. This class is used to configure the kernel mapping for a model. Nc6UbUO0Ul0UlX lgNkernel_mappingregistered_layer_namesuse_local_kernel)selfr>r@s r__init__KernelConfig.__init__es 0>0JnPR&(# 0r-c@SSKJn UXW"UUUS900URU'g)Nrr)r$r%revision)r&rr>)rAr$registered_namer%rr)rErs r update_kernelKernelConfig.update_kerneljs4+ o#)%0 O,r-cUR5H1up#[US5(dMURURU'M3 g)Nkernel_layer_name) named_moduleshasattrrJr?)rArnamemodules rstore_registered_layer_names)KernelConfig.store_registered_layer_namesws;!//1LDv2334:4L4L++D12r-cSnURU5 [UR[5(d$[ SUS[ UR535eURR 5Hup4X0RR5;a[ SUS35e[U[5(aSU;dSU;a[ SUS U35eMe[U[5(aaUR 5HKupVUS ;a[ S U35e[U[5(aSU;dSU;dM<[ SUS U35e M[ S USU35e g )a Validates the kernel_mapping to ensure that: 1. Each layer_name in the mapping is registered in the model (i.e., the model contains a module with a matching kernel_layer_name). 2. Each kernel value is either a string of the form 'org/repo:layer_name' or a dict mapping device types ("cuda", "rocm", "xpu", "npu") to such strings. 3. Each device key in a dict is one of "cuda", "rocm", "xpu", or "npu". 4. Each repo_name is a valid repository and layer name in the format 'org/repo:layer_name' (i.e., a string containing both a slash and a colon). 5. If a local path is detected, it should be in the format '/abs/path:layer_name'. The absolute path must include the `package_name`, like "/home/user/layer_norm". Args: model: The model instance whose modules are checked for registered kernel_layer_name attributes. Raises: ValueError: If a layer_name is not registered in the model, if a device is not supported, or if a repo_name is not a valid 'org/repo:layer_name' string. a For single device form remote { "RMSNorm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, For multiple devices form remote { "RMSNorm": { "cuda": "kernels-community/layer_norm:LlamaRMSNorm", "rocm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, ... } For single device form local { "RMSNorm": "/abs/path:LlamaRMSNorm", ... }, For multiple devices form local { "RMSNorm": { "cuda": "/abs/path:LlamaRMSNorm", "rocm": "/abs/path:LlamaRMSNorm", ... }, ... } z7Kernel mapping must be a dict of the following format: z, got: zLayer z[ is not registered in the model, please register it first using use_kernel_forward_from_hubr1r"zKernel mapping for 'zl' must be a valid repo name with a layer name (e.g., 'org/repo:layer_name' or '/abs/path:layer_name'), got: rr!z'Kernel mapping must follow the format: N) rO isinstancer>dictr ritemsr?valuesstr)rArMAPPING_FORMATr%kernelrr(s rsanitize_kernel_mapping$KernelConfig.sanitize_kernel_mapping|s # H ))%0$--t44I.IYY`aefjfyfyaz`{| #'"5"5";";"= J!JZZabhai!jkk-#>r-c LSSKJn 0n[U5nURR 5HupgUR (a UR O URnU(aXR-n[U[5(a1Un UR(d[XeXU5 Mz[XeXU5 M[U[5(dMUR 5H9upX:waM UR(d[XjXU5 M,[XjXU5 M; M X@lg)a Transforms a simple kernel_mapping of the form: { "RMSNorm": "kernels-community/layer_norm:LlamaRMSNorm", ... }, or for local path: { "RMSNorm": "/home/user/liger_kernels:LigerRMSNorm", ... }, into a nested mapping: { "RMSNorm": { "cuda": { Mode.INFERENCE: LayerRepository( repo_id="kernels-community/layer_norm", layer_name="LlamaRMSNorm", ) } } } or for local path: { "RMSNorm": { "cuda": { Mode.INFERENCE: LocalLayerRepository( repo_path=Path("/home/user/liger_kernels"), package_name="liger_kernels", layer_name="LigerRMSNorm", ) } } } that's compatible with the kernels library. The device is inferred from the model's parameters if not provided. The Mode is inferred from the model's training state. r)ModeN)r&r\rr>rTtrainingTRAINING INFERENCE TORCH_COMPILErRrVr@r,r7rS) rArcompiler\r*current_devicer%rXr)r(rs rcreate_compatible_mapping&KernelConfig.create_compatible_mappingsb !%e,"&"5"5";";"= J$)NN4==D000&#&&" ,,":yPbc(YVhiFD)))/%F/ 00&z9L^_,ZRde *8#>*1r-r=)NFr<)F) __name__ __module__ __qualname____firstlineno____doc__rBrGrOrYrc__static_attributes__r-rr9r9`s%1  M QlfJ1r-r9N)rrrr,r7r9rkr-rrls(#%P"*y1>y1r-