>:j.SSKJr SSKJr SSKrSSKJr SSKJrJrJ r SSK J r SSK J r SS KJrJr "S S \5rg) ) annotations)OptionalN)nn) BaseTunerBaseTunerLayercheck_target_module_exists)4TRANSFORMERS_MODELS_TO_ADAMSS_TARGET_MODULES_MAPPING) AdamssConfig) AdamssLayerLinearc^\rSrSr%SrSrS\S'\4r\ r SSU4Sjjjr \ S5r SSjrSS jrSS jrSS jrSS jrSS jrSSjrSrU=r$) AdamssModela@ Creates Adamss (Adaptive Multi-Subspaces) model from a pretrained model. The method decomposes weight matrices using SVD and clusters the decomposed space into multiple trainable subspaces for parameter-efficient fine-tuning. Args: model (`torch.nn.Module`): The model to be adapted. config (`AdamssConfig`): The configuration of the Adamss model. adapter_name (`str`): The name of the adapter, defaults to `"default"`. Returns: `torch.nn.Module`: The Adamss model. Example: ```python >>> from transformers import AutoModelForImageClassification >>> from peft import AdamssConfig, get_peft_model >>> config = AdamssConfig( ... r=500, ... num_subspaces=5, ... target_modules=["query", "value"], ... ) >>> model = AutoModelForImageClassification.from_pretrained("google/vit-base-patch16-224") >>> adamss_model = get_peft_model(model, config) ``` **Attributes**: - **model** ([`~torch.nn.Module`]) -- The model to be adapted. - **peft_config** ([`AdamssConfig`]): The configuration of the Adamss model. adamss_strprefixc2>0Ul[TU] XX4US9 g)N)low_cpu_mem_usage state_dict)_asa_total_subspacessuper__init__)selfmodelconfig adapter_namerr __class__s U/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/tuners/adamss/model.pyrAdamssModel.__init__Fs"%'!  fpqc[X5$)z5Helper to check if target module matches the pattern.)r) adamss_configkeys r_check_target_module_exists'AdamssModel._check_target_module_existsMs*-==r!c Uc [S5e[U[5(aURUURUR UR URURURS9 X R;a%URURURS9 ggUR"XU4SUR0UD6nURX!U5 URXTX5 g)z> Create and replace target module with Adamss-adapted module. NzCurrent Key shouldn't be `None`)inference_moder() ValueError isinstancer update_layerr num_subspaces subspace_rank init_weightsuse_asar(active_adapters set_adapter_create_new_module_record_asa_total_subspaces_replace_module) rr#rtarget target_nameparent current_keyoptional_kwargs new_modules r_create_and_replaceAdamssModel._create_and_replaceRs  >? ? fk * *   ++++**%%,;;  #7#77""4#7#7 HdHd"e8 00VDQD`D`dsJ  , ,\* U  j Ir!c[US5(dgURRUS5nUbUS::agURRUS5nXT-URU'g)zITrack total subspaces per adapter so the ASA schedule matches adamss_pkg.r-Nr)hasattrr-getr)rrr#modulelayer_num_subspaces prev_totals rr4'AdamssModel._record_asa_total_subspaces{sgv// $2266|QG  &*=*B ..22<C 2<2R!!,/r!c |[U[5(aUR5nOUn[U[RR 5(a][ UU4UR URURURURURURS.UD6nU$[SUS35e)z= Create a new Adamss module based on the target module type. )r,r-r.r/r0use_dynamic_rank svd_thresholdzTarget module zB is not supported. Currently, only `torch.nn.Linear` is supported.)r*rget_base_layertorchrr r,r-r.r/r0rFrG TypeError)rr#rr6kwargstarget_base_layerr;s rr3AdamssModel._create_new_modules fn - - & 5 5 7  &  ' 9 9  //+99+99*77%--!.!?!?+99  J"  (jk r!cURGHDnURUnUR(dM&URUs=:*=(a UR:*Os nUR R 5Vs/sH-n[U[5(dMX%R;dM+UPM/ nnU(dMU(a/UH)nURX#RUR5 M+ XR-S:HnU(dMU(dGMURX5n U bURXU5 UHnUR!U5 M GMG gs snf)uK Update importance scores and apply ASA masking (if enabled). This method should be called in **every** training step after ``loss.backward()`` and before ``optimizer.zero_grad()`` when ASA is enabled. Internally it: 1. Accumulates importance scores via EMA every step during the warmup period. 2. At mask intervals, applies global top-K masking and resets importance. This is the single entry point for ASA – using the :class:`AdamssAsaCallback` with HuggingFace ``Trainer`` simply delegates to this method. For custom training loops, call this directly instead of the callback. Args: global_step (`int`): The current training step. Example:: for step, batch in enumerate(dataloader): loss = model(**batch).loss loss.backward() optimizer.step() model.base_model.update_and_allocate(step) optimizer.zero_grad() rN)r1 peft_configr0 init_warmup final_warmuprmodulesr*r exp_avg_ipt_Aupdate_importanceasa_importance_betaasa_uncertainty_beta mask_interval_schedule_threshold_global_mask_to_targetreset_importance) r global_steprr within_warmupm asa_layersrAis_mask_intervalcurrent_targets rupdate_and_allocateAdamssModel.update_and_allocates),!00L%%l3F>>"..+TTATATTM ::--//a:a3MR^bqbqRq/ (F,,\;U;UW]WrWrs) +-A-AAQF }!1!1!%!9!9+!N!-//jY)F++L9)71s8E E&EcURR[[UR5S5S5nUS:XaUR 5nUS:XagXR :agXR::awSXR - URUR - - - n[S[SU55n[USS5n[URX2R- XE---5$UR$)zDHv22e>Y>Y6Y^g^q5rrs s.. .r!cSnURR5H\n[U[5(dMUR(dM-U[ [ URR555- nM^ U$)zAGet total number of subspaces from model (sum across all layers).r)rrRr*r r-rfrgvalues)rrorAs rrh$AdamssModel._get_asa_total_subspacess`jj((*F&+..63G3G3Gd6#7#7#>#>#@ABB+ r!c/nUHnURRUS5nURUnURUnURUn UR Un [ U5HQn X{bXcMX{X-R5XX-R5-n URX[U 45 MS M U(dg[R"UV s/sHoSPM sn 5nU[U5:a [S5nO*[R"U*U5SR5*nUHunnn [U 5U:nURUnUR UnU[U5:aUUUlU(d SUUlU[U5:dMnUUUlU(aMSUUlM gs sn f)z Apply **global** top-K masking across all layers. Collects importance scores from every subspace in every layer, ranks them globally, and keeps only the top ``target_subspaces`` active. rNz-inf)r-r@rS exp_avg_ipt_B exp_avg_unc_A exp_avg_unc_BrangemeanappendrIstacklenfloatkthvalueitemadamss_Aadamss_B requires_gradgrad)rtarget_subspacesrr^subspace_scoresrA n_subspacesipt_Aipt_Bunc_Aunc_Biscores all_scores thresholdidx is_activeparam_Aparam_Bs rrY"AdamssModel._global_mask_to_targets(* F ..22<CK((6E((6E((6E((6E;'8#ux'7EH,224588K7Q7Q7SS&&5'9: (! [[!@1A$!@A s?3 3f I 5EFqINNPPI#2 FCe y0Iool3Gool3GS\!-6 * (,GCL%S\!-6 * y(,GCL%#2"As G )r)FN)rboolrzOptional[dict]returnNone) r#r rrr6 nn.Moduler7rr8rr9r)rrr#r rAr rr)r#r rrr6rrr)r[rlrr)rnrlrz Optional[int])rrl)rrlrrr^listrr)__name__ __module__ __qualname____firstlineno____doc__r__annotations__r tuner_layer_clsr target_module_mappingr staticmethodr%r<r4r3rarXrhrY__static_attributes__ __classcell__)rs@rrrs DFC"nOPjnr>BrXfr rr>>'J#'J'J 'J  'J  'J'JRS!#!! !  !F2:h/$,-,-r!r) __future__rtypingrrIrpeft.tuners.tuners_utilsrrr peft.utilsr rr layerr r rr!rrs4# ZZ!&C-)C-r!