>:j"+SSKJr SSKrSSKrSSKJr SSKJr SSKJ r J r SSK J r J r SSKJr SSKJr SS KJr S S KJr S S KJrJr "S S\ 5rg)) annotationsN)Conv1D)is_bnb_4bit_availableis_bnb_available) BaseTunerBaseTunerLayer)3TRANSFORMERS_MODELS_TO_PVERA_TARGET_MODULES_MAPPING) BufferDict) _maybe_include_all_linear_layers) PveraConfig)Linear PveraLayerc|^\rSrSr%SrSrS\S'\r\ r S Sjr SSjr SSjr SU4S jjrS r\S 5rS rU=r$) PveraModel#a Creates Probabilistic Vector-based Random Matrix Adaptation (PVeRA) model from a pretrained transformers model. Args: model ([`~transformers.PreTrainedModel`]): The model to be adapted. config ([`PveraConfig`]): The configuration of the PVeRA model. adapter_name (`str`): The name of the adapter, defaults to `"default"`. low_cpu_mem_usage (`bool`, `optional`, defaults to `False`): Create empty adapter weights on meta device. Useful to speed up the loading process. Returns: `torch.nn.Module`: The PVeRA model. Example: ```py >>> from transformers import AutoModel >>> from peft import PveraConfig, get_peft_model >>> base_model = AutoModel.from_pretrained("facebook/dinov2-base") >>> config = PveraConfig(r=128, sample_at_inference=False) >>> model = get_peft_model(base_model, config) ``` **Attributes**: - **model** ([`~transformers.PreTrainedModel`]) -- The model to be adapted. - **peft_config** ([`PveraConfig`]): The configuration of the PVeRA model. pvera_lambda_strprefixcURUR5nURX5n[X0R5nSnURR 5HupVUR X55(dM[ U[R5(aURUR4nOg[ U[5(aP[URS5(aURROURRnUSSS2nOMUcUnMXt:wdM[!S[#XG555nM Uc Sn[%U5eU$)z Finds the largest input and output dimensions across linear layers that have been wrapped with PVeRA. This will be used for determining the size of the shared pvera_A and pvera_B matrices. Nds_shapec3<# UHup[X5v M g7fN)max).0abs T/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/tuners/pvera/model.py 'PveraModel._find_dim..bs%]<\DAc!ii<\sz\No layers types compatible with PVeRA were found. Please check `peft_config.target_modules`.)get_model_configmodel_prepare_adapter_configr named_modules_check_target_module_exists isinstancennr out_features in_featuresrhasattrweightrshapetuplezip ValueError) selfconfig model_config peft_config largest_shapekeymodule module_shapemsgs r _find_dimPveraModel._find_dimEs" ,,TZZ8 226H 6{JJO  ::335KC33KEE&")),,%22F4F4FF FF++9@PZ9[9[v}}55agananatat +DbD1 $ , , %%]C <\%] ] #6&  pCS/ !c@URU5up4[0URS9Ul[0URS9Ul[ R "SS9RUR5n[ RRR[ R"URS-U5US9n[ RRR[ R"X1R5US9nX`RU'XpRU'g)N) persistentcpu)devicer ) generator)r;r save_projectionpvera_Apvera_Btorch Generator manual_seedprojection_prng_keyr)initkaiming_uniform_emptyr)r2r3 adapter_namelinear_out_dim linear_in_dimrBrDrEs r _init_pvera_A_pvera_B PveraModel._init_pvera_A_pvera_Bjs(,v(>%""1G1GH !"1G1GH OO51==f>X>XY ((--00VXX\=1Yen0o((--00^XX1Vbk0l%, \"%, \"r=c&URX#5 gr)rQ)r2r$r3rNs r _pre_injection_hookPveraModel._pre_injection_hookys ""68r=c>[TU]U5 URR5HJnX!LaM URUR:wdM%[ SUR<SURS35e URR5Vs1sHoR iM nn[U5S:a[ SU35egs snf)z A helper method to check the config when a new adapter is being added. Raise a ValueError if there is something wrong with the config or if it conflicts with existing adapters. z`PVeRA PRNG initialisation key must be the same for all adapters. Got config.projection_prng_key=z but previous config had .r zdPVeRA projection weights must be saved for all adapters or none, but got multiple different values: N)super_check_new_adapter_configr5valuesrIr1rClen)r2r3existing_configsave_project_unique_values __class__s r rY$PveraModel._check_new_adapter_config|s )&1#//668O(22f6P6PP w\b\v\v[xy++:+N+N*OqR 9LPK[K[KbKbKd%eKd&<&? ? NNvv&B6;;d+B)77*99(55%djj2EuM%djj2EuM  v ff % %     **))&00  00llDLL,P[_eJ#6#66))%0  j Ir=c t[5(a SSKnSSKJn [ 5(aSSKJn UR SS5n URSS5n URSS5n [U[5(aUR5n OUn U (a[U WRR5(aaUR5nURU RRU RR U R"S .5 W"XCX40UD6$U (a[U WRR 5(aaUR5nURU R$U R&R(U R&R*S .5 W "XCX40UD6$[U [,RR.5(a-US (a"[0R2"S 5 S=US 'UlOV[U [65(a2S US'US (d"[0R2"S5 S =US 'UlO[9SUS35e[UR:[<5(a UR:nOUR:RUS5n[/UUUU4U UR>US.UD6nU$)Nrr ) Linear8bitLt) Linear4bitraFrgrh)has_fp16_weights thresholdindex) compute_dtypecompress_statistics quant_typerezjfan_in_fan_out is set to True but the target module is `torch.nn.Linear`. Setting fan_in_fan_out to False.Tis_target_conv_1d_layerzafan_in_fan_out is set to False but the target module is `Conv1D`. Setting fan_in_fan_out to True.zTarget module z is not supported. Currently, only the following modules are supported: `torch.nn.Linear`, `transformers.pytorch_utils.Conv1D`.)rarisample_at_inference) r bitsandbytesbnbr{rr|popgetr(rget_base_layerr)copyupdatestater}r~rrr-rrrFrwarningswarnrerr1rboolri)rprDrErNrqrtrvrr{r|rargrhtarget_base_layereightbit_kwargsfourbit_kwargsmodule_sample_at_inferencerws r rlPveraModel._create_new_modules    & ) " " 'zz&%($4e<$4e< fn - - & 5 5 7  &  j):CFFO>O P P#[[]N  ! !%6%D%D+<+C+C+W+W"3":":"E"E fGWW W )588?? ; ;&' 7JON'(<+F )6 2 204F, -*+ wJNM'(<+F )JJ  l66 = =)5)I)I &)5)I)I)M)Mk[`)a &      ",, :    r=)rDrE)returnztuple[int, int])r3rrNrrNone)r$z nn.Moduler3rrNrrr)r3rrr)__name__ __module__ __qualname____firstlineno____doc__r__annotations__rtuner_layer_clsr target_module_mappingr;rQrTrYrx staticmethodrl__static_attributes__ __classcell__)r^s@r rr#sQ:"FC! OO#J -96*JXKKr=r) __future__rrrFtorch.nnr)transformers.pytorch_utilsrpeft.import_utilsrrpeft.tuners.tuners_utilsrr peft.utilsr _buffer_dictr tuners_utilsr r3rlayerrrrr=r rs=# -E>&;%llr=