>:j@rSSKJr SSKrSSKJr SSKJrJr SSKJr SSK J r SSK J r J r "S S \5rg) ) annotationsN)nn) BaseTunerBaseTunerLayer)2TRANSFORMERS_MODELS_TO_LILY_TARGET_MODULES_MAPPING) LilyConfig) LilyLayerLinearcP\rSrSr%SrSrS\S'\r\ r Sr \ S5r S SjrS rg ) LilyModelaj Creates a Low-Rank Interconnected Adaptation Across Layers (Lily) model from a pretrained transformers model. The method is described in detail in https://arxiv.org/abs/2407.09946. Args: model ([`torch.nn.Module`]): The model to be adapted. config ([`LilyConfig`]): The configuration of the Lily model. adapter_name (`str`): The name of the adapter, defaults to `"default"`. Returns: `torch.nn.Module`: The Lily PEFT model. **Attributes**: - **model** ([`~transformers.PreTrainedModel`]) -- The model to be adapted. - **peft_config** ([`LilyConfig`]): The configuration of the Lily model. lily_strprefixc `Uc [S5e[U[5(aGURUURUR UR URURS9 gURXU5nX R;aURS5 URXTX5 g)z Create a new Lily layer with independent A and B for each layer. Sharing of A/B across layers is deferred to _post_injection_hook. NzCurrent Key shouldn't be `None`)scalingstride_Anum_B init_weightsF) ValueError isinstancer update_layerrrrrr_create_new_moduleactive_adaptersrequires_grad__replace_module) self lily_config adapter_nametarget target_nameparent current_keyoptional_kwargs new_modules S/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/tuners/lily/model.py_create_and_replaceLilyModel._create_and_replace5s  >? ? fi ( (    #++$--!''(55  00FSJ#7#77))%0  j Ic V[U[5(aUR5nOUn[U[RR 5(aA[ UUUR URURURURS9$[S[U5S35e)N)rrrrrz-Lily does not support target modules of type z yet.) rrget_base_layertorchrr rrrrrNotImplementedErrortype)r r!r"target_base_layers r(rLilyModel._create_new_moduleUs fn - - & 5 5 7  &  ' 9 9--#++$--!''(55 "$QRVWhRiQjjo"pqqr+cURn0nUR5Hupg[U[5(dMX7R;aM-UR 5nUR Rn Sn [UR[5(a URn O-URHn URU 5(dMU n M U cMX4n X;a/X\'X\RU5 M UR5HsuupnUSRUn[U5HJunnUS:waUURU'UU-U-nUU:wdM+UURUURU'ML Mu g)aC After all layers have been independently initialized, apply A/B sharing across layers. A sharing: for each (target_module_suffix, weight_shape) group, consecutive blocks of `stride_A` layers share the same A. The first layer in each block keeps its own A; subsequent layers in the block have their lily_A replaced by the group leader's. B sharing: all layers in the same (target_module_suffix, weight_shape) group share the B from the very first layer in that group. Both lily_A and lily_B are nn.Linear modules, so they move correctly with model.to(device) via standard nn.Module parameter propagation. Note: (target_module_suffix, weight_shape) is used as the group key rather than target_module_suffix alone, to correctly handle architectures like UNet where the same target key can appear with different shapes across layers. Nr)r named_modulesrr lily_Ar-weightshapetarget_modulesrendswithappenditemslily_B enumerate)rmodelconfigr!rtarget_to_layerskeymodulebaser7matched_suffixsuffix group_key target_suffixlayersshared_Bilayer group_idxs r(_post_injection_hookLilyModel._post_injection_hookisq$?? KM ..0KCfi00==0((*DKK%%E"N&//55!'!6!6$33F||F++)/4%'/I0.0 +  ' . .v 6318/?.D.D.F * "]Fay'' 5H%f-5619ELL.(]h6  >17 1B1I1I,1WELL../Gr+N)r>z nn.Moduler?r r!rreturnNone)__name__ __module__ __qualname____firstlineno____doc__r__annotations__r tuner_layer_clsrtarget_module_mappingr) staticmethodrrM__static_attributes__rOr+r(r r sB$FCONJ@rr&@Xr+r ) __future__rr.rpeft.tuners.tuners_utilsrr peft.utilsrr?r rKr r r rOr+r(r_s1# J$KX KXr+