>:j3SSKrSSKJr SSKrSSKJr SSKJs Jr SSK J r SSK J r J r SSKJr SSKJr "SS \ 5r"S S \R&\5rg) N)Optional)Conv1D)BaseTunerLayercheck_adapters_to_merge) transpose) BufferDictc \rSrSrSrSrS\R4Sjr\ S\ 4Sj5r SS\ S \ S \ S \ 4S jjrSS \ 4S jjrSrg) PveraLayer)pvera_lambda_bpvera_lambda_d)pvera_Apvera_B base_layerc jXl0Ul[R"05Ul[R "05Ul[R "05UlSUlSUl SUl /Ul UR5n[U[R5(aURUR pCO^[U["5(aI[%UR&S5(aUR&R(OUR&R*up4WUlWUlX lg)NFds_shape)rrnn ModuleDict pvera_dropout ParameterDictr rrr_disable_adaptersmerged_adaptersget_base_layer isinstanceLinear in_features out_featuresrhasattrweightrshapekwargs)selfrr#rrs T/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/tuners/pvera/layer.py__init__PveraLayer.__init__"s$]]2.!..r2 ..r2.2 -1 "'!((* j")) , ,(2(>(> @W@W  F + +.5j6G6G.T.T !!**ZdZkZkZqZq &K'( returnc,[UR5$N)boolr)r$s r%mergedPveraLayer.merged@sD(())r(rr d_initialinference_modec  XUS::a[SU35eX@RU'US:a[R"US9n O[R"5n UR R [R"X055 [R"[R"UR5SS9URU'[R"[R"US-5SS9URU'X lX0lX;Ga[#UR5S:a [S 5e[%URR'55Sn [%UR R'55Sn S n U R(SUR*:a3[U R-S U R(SUR*55eU R(SUR:a3[U R-S U R(SUR55eS n U R(SURU:a6[U R-S U R(SURU55eU R(SURU:a6[U R-S U R(SURU55eXRU'XR U'U(aUR/XS9 UR1U5 UR3UR4US9 g)Nrz?`r` should be a positive integer value but the value passed is )pT) requires_gradrz`The `pvera_A` and `pvera_B` buffers are empty. This should not happen. Please report this issue.z{} has a size of {} but {} or greater is required; this probably happened because an additional PVeRA adapter was added after the first one with incompatible shapes.rrz{} has a size of {} but {} or greater is required; this probably happened because an additional PVeRA adapter with a lower rank was added after the first one; loading the adapters in reverse order may solve this.r/)r0) ValueErrorrrDropoutIdentityrupdater Parametertorchonesrr randnrrrlenlistvaluesr"rformatreset_pvera_parameters%_move_adapter_to_device_of_base_layer set_adapteractive_adapters)r$ adapter_namerrrr init_weightsr/r0r#pvera_dropout_layer pvera_A_param pvera_B_param error_tmpls r% update_layerPveraLayer.update_layerDs 6^_`^abc c | 3 "$**}"= "$++-  !!"--0S"TU,.LLDDUDU9Vfj,kL),.LLQU9K[_,`L)   &4<< 1$ v!!4!4!67:M !4!4!67:MR  ""1%(8(88 !2!29m>Q>QRS>TVZVfVf!ghh""1%(9(99 !2!29m>Q>QRS>TVZVgVg!hii3  ""1%|(<< !2!29m>Q>QRS>TVZV\V\]iVj!kll""1%|(<< !2!29m>Q>QRS>TVZV\V\]iVj!kll)6LL &)6LL &   ' ' ' J 22<@ --nMr(clXRR5;a[R"5 [R R URU5RU5 [R R URU5 SSS5 gg!,(df  g=fr+) rkeysr<no_gradrinitzeros_fill_r )r$rGr/s r%rC!PveraLayer.reset_pvera_parametersss ..335 5t22<@AGG Rt22<@A! 6 s A(B%% B3) rrrr#rrrrrr rrN)皙?F)rV)__name__ __module__ __qualname____firstlineno__adapter_layer_namesother_param_namesrModuler&propertyr,r-r floatrMrC__static_attributes__r(r%r r s>.299<***$BNBN BNBNBNHBeBBr(r c^\rSrSrSS\S\S\S\S\S\S\S \S \S \S S 4U4Sjjjr SS\S\ \ \S S 4Sjjr SSjr S \R4SjrSrS\RS \R4SjrS \4U4SjjrSrU=r$)rrrrGrrfan_in_fan_outis_target_conv_1d_layerrHr/sample_at_inferencer)Nc >[[RU]5 [R"X40U D6 XplXlX@lURXBX5XiU S9 Xl g)Nr6) superrrr&r rdrf_active_adapterrMre)r$rrrrGrrrdrerHr/rfr# __class__s r%r&Linear.__init__sY bii')D77,#6 + ,]dmn'>$r( safe_merge adapter_namesc>[X5nU(dgUGHnX0RR5;dM#UR5nU(aURR R 5nXPRU5- n[R"U5R5(d[SUS35eXTRlO.UR=R URU5- slURRU5 GM g)a Merge the active adapter weights into the base weights Args: safe_merge (`bool`, *optional*): If True, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to `False`. adapter_names (`List[str]`, *optional*): The list of adapter names that should be merged. If None, all active adapters will be merged. Defaults to `None`. Nz1NaNs detected in the merged weights. The adapter z seems to be broken)rrrPrr!datacloneget_delta_weightr<isfiniteallr7rappend)r$rlrmactive_adapterr orig_weightss r%merge Linear.merges0D  +N!4!4!9!9!;;!002 $.#4#4#9#9#?#?#AL $9$9.$IIL >>,7;;==(OP^O__rs.:%%*%%**d.C.CN.SS*$$++N;%,r(cUR(d[R"S5 g[UR5S:aURR 5nXR R5;a<UR5R=RURU5-sl [UR5S:aMgg)Nz Already unmerged. Nothing to do.r) r-warningswarnr?rpoprrPrr!rorq)r$rus r%unmergeLinear.unmerges{{ MM< = $&&'!+!11557N!4!4!9!9!;;##%,,11T5J5J>5ZZ1$&&'!+r(cURUnURUnURnURnURS:H=(a- U[ R :H=(d U[ R:HnURUnURUnU(a@UR5nUR5nUR5nUR5nUSS2SUR24RUR5n USUR2SS24RUR5n URS5nURS5nUSURS5S-2SS24nU SU RS5S-2SS24n [!X-Xy--UR"5n U(aU RUS9n U $)z Compute the delta weight for the given adapter. Args: adapter (str): The name of the adapter for which the delta weight should be computed. cpuNrr)dtype)rrdevicertyper<float16bfloat16rr r_rtor unsqueezesizerrd) r$adapterrrrr cast_to_fp32lambda_dlambda_bsliced_Asliced_B output_tensors r%rqLinear.get_delta_weights,,w',,w'  {{e+c%--1G1b5TYTbTbKb &&w/&&w/ mmoGmmoG~~'H~~'H10 0 000144X__E.T...1255hooF%%b)%%b)3hmmA.!33Q673hmmA.!33Q67!8#68;N"OQUQdQde ),,5,9Mr(cUR(dUR(d?U(a8[R"SU-5n[R"U5nXU--nU$UnU$)Ng?)trainingr<exp randn_like)r$mulogvarrfstdepszs r%_reparametrizeLinear._reparametrizesR ==3F))C&L)C""3'C3YAAr(xc  URnUR(a9UR(aUR5 UR"U/UQ70UD6nGOUR(aUR"U/UQ70UD6nGOrUR"U/UQ70UD6nUR GHKnX`R R5;aM#UR UnURUnURUn URUn U SS2SUR24RUR5n U SUR2SS24RUR5n URUn URUR5nU[ R""U "U5U 5-R%SSS9upXX[ R""UR'XUR(5U 5--nGMN URU5nU$)Nrr)dim)rdisable_adaptersr-r}rrFrrPr rrrrrrrFlinearchunkrrf)r$rargsr#previous_dtyperesultrurrrrrrdropoutrrs r%forwardLinear.forwards  {{ __Q888F [[__Q888F__Q888F"&"6"6!)<)<)A)A)CC..~>..~>,,~6,,~6 #1&8(8(8&8#89<* r(c*>[TU]5nSU-$)Nzpvera.)rh__repr__)r$reprjs r%rLinear.__repr__4sg #~r()rirdrerf)rr2FFTrVF)FN)r)N)rWrXrYrZr strintr_r,r&rr@rwr}r<Tensorrqrrrr` __classcell__)rjs@r%rrs"$(-!$)?? ?  ?  ???"&???"? ??2$<$rsJ -L&%oBoBdgRYY gr(