>:j2SSKJr SSKrSSKrSSKrSSKrSSKrSSKrSSKJ r SSK J r SSK J r SSKJrJrJr SSKrSSKrSSKrSSKJr SSKJrJr SS KJrJr SS KJr SS KJrJ r SS K!J"r" SS K#J$r$J%r% SSKJ&r& SSK'J(r(J)r)J*r* SSK+J,r,J-r-J.r.J/r/J0r0J1r1J2r2J3r3J4r4J5r5J6r6J7r7J8r8J9r9J:r:J;r;Jr>J?r?J@r@JArAJBrBJCrCJDrDJErEJFrFJGrGJHrHJIrIJJrJJKrKJLrLJMrMJNrNJOrO SrP\"R"\R5\"R"S5:a SSKJSrS \S"5rP/SQrTS7SjrUS8SjrVS9SjrWS:SjrX"SS\RR5r["SS\[5r\"SS\[5r]S;S jr^S!r_S"r`S#raS<S=S$jjrbS>S?S%jjrcS&rdS@S'jreS(rfS)rgSAS*jrhSBS+jriSCS,jrjS;S-jrkSDS.jrlS/rmSES0jrnSFS1jroSGS2jrpSHS3jrqSISJS4jjrrS5rsSKS6jrtg)L) annotationsN)Sequence) nullcontext) attrgetter)AnyOptionalUnion)FullyShardedDataParallelPlugin)add_hook_to_moduleremove_hook_from_module)is_npu_availableis_xpu_available) file_exists)EntryNotFoundErrorHFValidationError)version) storage_ptr storage_size)PreTrainedModel)is_gptqmodel_availableis_torch_tpu_availableis_transformers_ge_v5_1_0)$ CONFIG_NAMEEMBEDDING_LAYER_NAMESINCLUDE_LINEAR_LAYERS_SHORTHANDSAFETENSORS_WEIGHTS_NAME5TRANSFORMERS_MODELS_TO_ADALORA_TARGET_MODULES_MAPPING4TRANSFORMERS_MODELS_TO_ADAMSS_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_BOFT_TARGET_MODULES_MAPPING1TRANSFORMERS_MODELS_TO_C3A_TARGET_MODULES_MAPPING4TRANSFORMERS_MODELS_TO_DELORA_TARGET_MODULES_MAPPING7TRANSFORMERS_MODELS_TO_FOURIERFT_TARGET_MODULES_MAPPING5TRANSFORMERS_MODELS_TO_GRALORA_TARGET_MODULES_MAPPING1TRANSFORMERS_MODELS_TO_HRA_TARGET_MODULES_MAPPING6TRANSFORMERS_MODELS_TO_IA3_FEEDFORWARD_MODULES_MAPPING1TRANSFORMERS_MODELS_TO_IA3_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_LILY_TARGET_MODULES_MAPPING6TRANSFORMERS_MODELS_TO_LNTUNING_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_LOHA_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_LOKR_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_MISS_TARGET_MODULES_MAPPING1TRANSFORMERS_MODELS_TO_OFT_TARGET_MODULES_MAPPING4TRANSFORMERS_MODELS_TO_PEANUT_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_POLY_TARGET_MODULES_MAPPING8TRANSFORMERS_MODELS_TO_PREFIX_TUNING_POSTPROCESS_MAPPING3TRANSFORMERS_MODELS_TO_PSOFT_TARGET_MODULES_MAPPING3TRANSFORMERS_MODELS_TO_PVERA_TARGET_MODULES_MAPPING6TRANSFORMERS_MODELS_TO_RANDLORA_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_ROAD_TARGET_MODULES_MAPPING3TRANSFORMERS_MODELS_TO_SHIRA_TARGET_MODULES_MAPPING6TRANSFORMERS_MODELS_TO_TINYLORA_TARGET_MODULES_MAPPING4TRANSFORMERS_MODELS_TO_VBLORA_TARGET_MODULES_MAPPING2TRANSFORMERS_MODELS_TO_VERA_TARGET_MODULES_MAPPING4TRANSFORMERS_MODELS_TO_WAVEFT_TARGET_MODULES_MAPPING WEIGHTS_NAME&bloom_model_postprocess_past_key_value*starcoder_model_postprocess_past_key_valueFz0.29.0)is_mlu_availablec>[RR5(ag[[RS5(a.[RR R5(ag[ (ag[5(ag[5(agg)Ncudampsmluxpunpucpu) torchrA is_availablehasattrbackendsrB mlu_availablerr M/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/utils/other.py infer_devicerOsg zz    ' 'ENN,>,>,K,K,M,M        rMc[USS5=(d [USS5n[USS5S:Hn[USS5S:Hn[USS5S:Hn[USS5S :Hn[USS5S :H=(d [US S5nUc0nUR5H upSU lM U(dU(dU(dU(dU(dUR5Hn U R[ R :Xd U R[ R:XdMAU RRS :wdM]U RR[ R5U l M U(d#U(dU(dU(dU(dU(aU(aS U;d US (aD[US5(aUR5 O"Sn UR5R!U 5 S[#[$R&"UR(5R5;n U (d*[+U5S:a[,R."S[05 U (d0OSU0n UR("S0U D6 U$)a Note this method only works for `transformers` models. This method wraps the entire protocol for preparing a model before running a training. This includes: 1- Cast the layernorm in fp32 2- making output embedding layer require grads 3- Add the upcasting of the lm head to fp32 4- Freezing the base model layers to ensure they are not updated during training Args: model (`transformers.PreTrainedModel`): The loaded model from `transformers` use_gradient_checkpointing (`bool`, *optional*, defaults to `True`): If True, use gradient checkpointing to save memory at the expense of slower backward pass. gradient_checkpointing_kwargs (`dict`, *optional*, defaults to `None`): Keyword arguments to pass to the gradient checkpointing function, please refer to the documentation of `torch.utils.checkpoint.checkpoint` for more details about the arguments that you can pass to that method. Note this is only available in the latest transformers versions (> 4.34.1). is_loaded_in_8bitFis_loaded_in_4bitquantization_methodNgptqaqlmeetqtorchaohqq hqq_quantized Params4bit use_reentrantenable_input_require_gradsc&URS5 g)NT)requires_grad_)moduleinputoutputs rNmake_inputs_require_gradAprepare_model_for_kbit_training..make_inputs_require_grads))$/rMgradient_checkpointing_kwargsrzgradient_checkpointing_kwargs is not supported in this version of transformers. The passed kwargs will be ignored. if you want to use that feature, please upgrade to the latest version of transformers.rL)getattrnamed_parameters requires_grad parametersdtyperGfloat16bfloat16 __class____name__datatofloat32rIr\get_input_embeddingsregister_forward_hooklistinspect signaturegradient_checkpointing_enablelenwarningswarn FutureWarning)modeluse_gradient_checkpointingrdloaded_in_kbitis_gptq_quantizedis_aqlm_quantizedis_eetq_quantizedis_torchao_quantizedis_hqq_quantizednameparamrb_supports_gc_kwargsgc_enable_kwargss rNprepare_model_for_kbit_trainingrs&U$7?m75ReglCmN'   eAA B M MB  #s+H'IA'M MMj *B0OQn/o  ++?.>? LrMcURUR5nUSS2SS24R5USS2SS24'X#SS2S4'Uc [S5eUR US:HU5 U$)a Shift input ids one token to the right. Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): input ids pad_token_id (`int`): The id of the `padding` token. decoder_start_token_id (`int`): The id of the `start` token. Nrrz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_) input_ids pad_token_iddecoder_start_token_idshifted_input_idss rNshift_tokens_rightrsz"++IOO<(CRC0668ae4adLMM""#4#[TU]5 XlU/UlSUl[ 5UlUR"U40UD6 UR"U40UD6 UR5 g)zEExtra kwargs will be passed to `self.init_modules` and `self.update`.FN) super__init__original_module_active_adapter_disable_adaptersset _adapters init_modulesupdate check_module)selfmodule_to_save adapter_namekwargsrls rNr!AuxiliaryTrainingWrapper.__init__s_ - ,~!& ,1&1 L+F+ rMc [e)zWA place to initialize PyTorch modules in `__init__` before the call to `self.update()`.NotImplementedErrorrrrs rNr%AuxiliaryTrainingWrapper.init_modules(!!rMc[ez:Return all adapter names that can be found on this module.rrs rN_get_available_adapters0AuxiliaryTrainingWrapper._get_available_adapters,rrMcg)aReturns a user friendly identifier for error messages, e.g. for type compatibility error messages from `check_module()` so that the user can backtrack where the error comes from. A generic "training wrapper" is less helpful than "modules_to_save", for example. ztraining wrapperrLrs rN_error_message_name,AuxiliaryTrainingWrapper._error_message_name0s "rMc[RR[RR[RR[RR 4n[ URU5(a4URRn[UR5SU35eSSK J n [ URU5(a4URRn[UR5SU35eg)z@Perform some sanity checks on the module to ensure that it worksz& cannot be applied to modules of type rBaseTunerLayerN) rGr ModuleDict ModuleList ParameterDict ParameterListrrrl TypeErrorrpeft.tuners.tuners_utilsr)rforbidden_classescls_namers rNr%AuxiliaryTrainingWrapper.check_module7s #XX00%((2E2EuxxG]G]_d_g_g_u_uv d**,= > >++55Ht779::`ai`jkl l < d**N ; ;++55Ht779::`ai`jkl l [TU]U5$![a Of=fSUR;a%[S[ U5R SUS35eURSnUR (dUR(d[URU5$URX5(aURX5$[S[ U5R SUS35e)N_modules'z' object has no attribute ') r __getattr__AttributeError__dict__typermrrrerrr)rrrrls rNr$AuxiliaryTrainingWrapper.__getattr__gs 7&t, ,    T]] * 1T$Z%8%8$99TUYTZZ[!\] ]-- +  )=)=4//6 6  " "4 1 1((7 7qd!4!4 55PQUPVVWXYYs  c ZXR;aURRU5 gg)aCalled when this instance should be part of an adapter's training. Adds the given adapter to the list of adapters that this instance is training along with. Additional kwargs are expected to be the same kwargs that are also passed for initializing this class. N)raddrs rNrAuxiliaryTrainingWrapper.updates% ~~ - NN  | , .rMc*[[RURR5nUR n0n[ R"UR5nUR5HnXeR;dMX6XF'M U"S0UD6nU$)zX Creates a new hook based on the old hook. Use it only if you know what you are doing ! rL) re acceleratehooksrlrmrrtrurkeysrh)rold_hook old_hook_cls old_hook_attrfiltered_old_hook_attrold_hook_init_signatureknew_hooks rN_create_new_hook)AuxiliaryTrainingWrapper._create_new_hooksz//1C1C1L1LM  )) !#")"3"3L4I4I"J##%A666,9,<&)& 9"89rMcURSS5nUcg[U5[U5:wa&S[U5S[U5S3n[U5eg)zMCheck if the arguments are compatible with the configs and state of the model adapter_namesNzNLength of `adapter_names` should be the same as the number of inputs, but got z and z respectively.)getrwr)rxargsrrmsgs rN_check_forward_args,AuxiliaryTrainingWrapper._check_forward_argssb ?D9   q6S' '`}%&eCF8>C S/ ! (rMc[errrrrrs rN_forward_wrapped)AuxiliaryTrainingWrapper._forward_wrappeds!!rMc[errrrrrrs rN_forward_wrapped_mixed_batch5AuxiliaryTrainingWrapper._forward_wrapped_mixed_batchs "!rMc[e)z\The forward call when no adapter is involved in the forward computation, only the base modelrrs rN_forward_wrapped_passthrough5AuxiliaryTrainingWrapper._forward_wrapped_passthroughrrMc [RR[RR[RR[RR [RR 4nSRUVs/sHofRPM sn5n[URU5(d[SUS35e[U5n/n UH9n U R[U5V V s/sHupX:XdM U PM sn n 5 M; [[!U55V s/sHn SPM nn [U5H`unnXUnUS:XaUR"U/UQ70UD6nOUR""UU/UQ70UD6n[X5Hun nUU UU'M Mb [R$"U5$s snfs sn n fs sn f)N, z# UHoTR;v M g7fr)r).0rrs rN 3AuxiliaryTrainingWrapper.forward..s'jUi't~~(EUis)rpopranyrr rr&)rrrrrs` rNforward AuxiliaryTrainingWrapper.forwards   4T4V4 ?D9  C'jUYUiUi'j$j$j44QHHH H  ((rBrJrMrPrV__static_attributes__ __classcell__rls@rNrrsX ,.-)++!#OY# """m$&&$$$$  Z0-  """"/2";>"JM" " " $! $*- $>G $SV $  $D Z *":,"C,""""'+"[TU]XUS9 g)N) tied_module)rr)rrrrvrls rNrModulesToSaveWrapper.__init__:s ;OrMc L[RR05Ulgr)rGrrrtrs rNr!ModulesToSaveWrapper.init_modules=s$xx2226rMcg)NrtrLrs rNr(ModulesToSaveWrapper._error_message_nameAs rMcUR(dUR"U/UQ70UD6$URURS"U/UQ70UD6$)Nr)rr rtrs rNr%ModulesToSaveWrapper._forward_wrappedDsQ##44QHHH H##D$8$8$;[T U]U5 [5nURR 5HfupVUR 5nUS:XdM[ US5(dM0SSKnURRURR5SS9n O XR;aU U(aT[RR"URR SS06n URU lXRU'O-["R$"UR5URU'SSS5 [ URUS5(a[URUR&n UR)U 5n [+URU5 [-URUU 5 URR/S5 XR0:Xa[3URUS5 gg!,(df  N=f)Nrds_numel) modifier_rankbiasF_hf_hookT)rrrrrfnumelrI deepspeedzeroGatheredParametersrhrtrGrrweightrcopydeepcopyrrr r r^rr) rrrvrcontext_managerr!r num_paramsr new_linearrrrls rNrModulesToSaveWrapper.updateVs |$%-,,==?HAJQ75*#=#= "+.."C"CDDXDXDcDcDeuv"C"w@ 33 3 !&+2D2D2J2J!WQV!WJ(3(:(:J%9C((69=tG[G[9\D((6! 4'' 5z B B++L9BBH,,X6H #D$8$8$F G t33LA8 L ++E2 .. . $T%9%9,%G N /)!s 0B G?? H c>[TU]U5 U(a-URHn[URUS5 M gURR 5Hn[US5 M g)zTakes care of setting the required_grad flag on the modules_to_save. If adapters are enabled, gradients for the modules_to_save are required as well. TFN)rr3rrrtvalues)rr2rr_rls rNr3$ModulesToSaveWrapper.enable_adapters{s_ (  $ 4 4 ()=)=l)KTR!5..557(78rMc*[U[5(aU$[U5S:Xa [S5eUVs/sHo"UR;dMUPM nn[U5S:a[SUS[U535eU(dSnU$USnU$s snfr6rz*Please specify at least one adapter to setrz*Only one adapter can be set at a time for z, got N)rrrwrrtrrnadapter_names_in_moduleadapter_name_to_sets rNr8&ModulesToSaveWrapper.check_set_adapters lC ( (  |  !IJ J.:"Xl4CWCW>W1l"X & '! +I$vVYZqVrUstu u'"& #"#:!"< ""#Ys B Bc[U[5(aU/n[U5S:a[SUS35eURHn[ UR US5 M [U5S:Xa/UlgUSnX@R;a[SUSUR35e[ UR UU(+5 X@lg) aSet the active adapter Additionally, this function will set the specified adapter to trainable (i.e., requires_grad=True) unless inference_mode is True. Args: adapter_names (list[str], str): The name(s) of the adapter(s) to set as active. inference_mode (bool, optional): Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False. rAttempted to set multiple (z') adapters at once for modules_to_save.FrNr;r<) rrrwrrrrtrr)rrr=currently_active_adapter_namers rNr> ModulesToSaveWrapper.set_adapters mS ) )*OM }  !:=/Ipqr r-1-A-A ) $T%9%9:W%XZ_ `.B }  "#%D $Q' ~~ -x ~^DNNCSTU U !5!5l!CEWX+rMcXR;ag[U[[45(a7[ U5S:a(UR R n[SUSUS35eXR;aURRU5 U(dURU /Ul gUSnX@R;aURU /Ul gX@RS:waURU5 URU g) Delete the adapter if present. This method will also set a new active adapter if the deleted adapter was the active adapter. It is important that the new adapter is chosen by the caller in a deterministic way, so that the same adapter is chosen on all layers. Nrr) adapters at once for , which is not allowed.r) rtrrstuplerwrlrmrrremoverrr>rrrArnew_active_adapters rNrB#ModulesToSaveWrapper.delete_adapters 33 3  )D%= 9 9cBU>VYZ>Z>>**D-.A-BBYZ^Y__vw  >> ) NN ! !, /"$$\2#%D 03 %9%9 9$$\2#%D  !5!5a!8 8   / 0   .rMcXR;a0$URUR5Vs0sH o"SUSU3_M sn$s snfNzmodules_to_save.rrrt state_dict)rrrs rNrP0ModulesToSaveWrapper.adapter_state_dict_load_mapsU ~~ -IBFBVBVWcBdBoBoBqrBqQ%l^1QC88BqrrrsAc XR;a0$URUR5Vs0sHnX2SUSU3_M sn$s snfrrrrrrs rNrM'ModulesToSaveWrapper.adapter_state_dictsd ~~ -I)),7BBD D ,\N!A3?@ @D   sA cURURn[US5(a&U(aURX#S9 UR 5nU$)zUnloading in case of `ModulesToSave` means to simply return the wrapped module. However, if the wrapped module is itself a tuner, we'll call merge on it before. base_layerrUr)rtrrIrTget_base_layer)rrTrUr new_modules rNrV7ModulesToSaveWrapper.unload_and_optionally_merge_modulesQ))$*=*=>  :| , ,  J T#224JrMcH[URR55$r)rrtrrs rNr,ModulesToSaveWrapper._get_available_adapterss4'',,.//rM)rrtrr]r^r`rarcrfrX)rmrhrirjrkrrlrrrrrr rrrr3r8r>rBrPrMrVrrnrorps@rNrrrr4so,@?P7!Q H8ER#OJ 8#2 ,D%/Ns  '+[TU]XX4S9 SUlg)N) token_indices tied_adapter)rrr)rrrrrrls rNrTrainableTokensWrapper.__init__"s! ]n $rMc.URR$r) token_adapterrrs rNr&TrainableTokensWrapper.original_module.s!!,,,rMcBSSKJn U"URXU5Ulg)Nr)TrainableTokensLayer)peft.tuners.trainable_tokensrrr)rrrrrs rNr#TrainableTokensWrapper.init_modules4sE2$2F2F eqrrMcg)Ntrainable_token_indicesrLrs rNr*TrainableTokensWrapper._error_message_name<s(rMc US:H$)NrrLrs rNr'TrainableTokensWrapper._hasattr_wrapped?s xrMc|US:Xa(USRURR5$[SUS35e)NrrzWThis code should've never been reached, probably a bad check in `_hasattr_wrapped` for zH. Please file an issue under https://github.com/huggingface/peft/issues.)get_merged_weightsrr RuntimeErrorrs rNr'TrainableTokensWrapper._getattr_wrappedBsQ 8 ?+>>t?Q?Q?a?ab befjeklU U  rMcrUR(dUR"U/UQ70UD6$URU5$r)rr rrs rNr'TrainableTokensWrapper._forward_wrappedNs7##44QHHH H!!!$$rMc:URRX/5$r)rforward_adaptersrs rNr3TrainableTokensWrapper._forward_wrapped_mixed_batchSs!!2216FGGrMc.UR"U/UQ70UD6$r)rrs rNr 3TrainableTokensWrapper._forward_wrapped_passthroughVs!!!5d5f55rMc |>XR;aURR"U40UD6 [TU]U5 gr)rr update_layerrr)rrrrls rNrTrainableTokensWrapper.update[s4  /    + +N Ef E ~&rMcJURR(a0$SSU30$)Nrz%token_adapter.trainable_tokens_delta.rrr7s rNrP2TrainableTokensWrapper.adapter_state_dict_load_mapcs+    * *I6:_`l_m8noorMc URR(a0$SVs0sHnSU3USUSU3_M sn$s snf)N)trainable_tokens_deltaztoken_adapter.rrrs rNrM)TrainableTokensWrapper.adapter_state_dicths]    * *I\v [uVWnQC *~aS,-P"Q Q[u   s>cZ>[TU]U5 URRU5 g)zkEnables/disables the underlying `TrainableTokens` adapter. Also handles the internal adapter disable flag. N)rr3r)rr2rls rNr3&TrainableTokensWrapper.enable_adaptersss& ( **73rMc>[U[5(aU$[U5S:Xa [S5eUVs/sH o"URR ;dMUPM" nn[U5S:a[SUS[U535eU(dSnU$USnU$s snfr)rrrwrrrrs rNr8(TrainableTokensWrapper.check_set_adapter{s lC ( (  |  !IJ J/;"ml4CUCUClCl>l1l"m & '! +I$vVYZqVrUstu u'"& #"#:!"< ""#ns BBcR>[TU]XS9 URRXS9 g)Nr=)rr>r)rrr=rls rNr>"TrainableTokensWrapper.set_adapters* MI &&}&TrMcURRU5 [U[[45(a7[ U5S:a(UR Rn[SUSUS35eXR;aURRU5 U(d/Ul gUSURR;a/Ul gUSnURU5 g)rrrrrNr)rrBrrsrrwrlrmrrrrrr>rs rNrB%TrainableTokensWrapper.delete_adapters )),7 )D%= 9 9cBU>VYZ>Z>>**D-.A-BBYZ^Y__vw  >> ) NN ! !, /"#%D  q !););)R)R R#%D 03 +,rMcvU(aURRX#S9 URR5$)zUnloading for `TrainableTokensWrapper` means to return the wrapped module, e.g. the embedding layer and, if requested, merging the `TrainableTokens` adapter onto the wrapped module. r)rrTrrSs rNrV9TrainableTokensWrapper.unload_and_optionally_merge_modules3     $ $ $ X!!0022rMc\[URRR55$r)rrrrrs rNr.TrainableTokensWrapper._get_available_adapterss"4%%<<AACDDrM)rrrr)rrgrrrz list[int]rYrbr]r^r`rarcrfrX)rmrhrirjrkrrlrrrmrrrrrrrr rrPrMr3r8r>rBrVrrnrorps@rNrrs,UT)ss $' $ $! $  $ $-- s)   % H6 'p  4#4UU-B33'+3 ?F))C.$K   %F ; &&rMcBURSRURS5SS55nURSRURS5SS55nURS5SnURU5nX#XT4$![a SnN7f=f)Nrr)rrrr)r{rIr grandparentrrs rN _get_submodules_with_grandparentrs  #))C."*=!> ?F))#((399S>#23F*GH ))C.$K   %F  33  s2B BBcRUR5Hup#X;dM SUlM g)NF)rfrg)r{rrps rN_freeze_adapterrs$&&(  #AO)rMc ^ SSKJn Uc[nU(dg/n [5n UR SS9V V s/sHupU PM n n n U GH@m [ U 4SjU55nU(dM"[ UT 5unnnn[UU5(a[SUS[U5S US 35e[UU5(a.UR"U40UD6 URURUS 9 O[UU5(a.UR"U40UD6 URURUS 9 OOU"UU40UD6nU(aURXS 9 OUR/US 9 [UUU5 U RU5 U RU5 GMC [U5R!U 5nU(aU (d[S US 35eU $s sn n f)aMWraps modules that are supposed to be re-trained either normally, i.e. marking them to require gradients and saving them alongside other modules, or with certain methods that go alongside PEFT methods, such as retraining specific token indices using selective read/write. Note that you need to validate beforehand if there are layers targeted by multiple wrappers, e.g. if the 'embedding' layer is configured for both `ModulesToSaveWrapper` and `TrainableTokensWrapper` there would be conflicts down the line. The default is to wrap the module in a `ModulesToSaveWrapper` wrapper. If `strict_module_check` is set, this method raises an ValueError, similar to BaseTuner.inject_adapter when none of the requested modules in `module_names` is not found in the model. The `active_adapter` flag indicates if this new adapter should be activated. rrNFremove_duplicatec3F># UHnTRU5v M g7fr)endswith)r* target_keyrIs rNr+!_set_trainable..s!Z\z#,,z":":\s!z'You are trying to target a module with z that is a child of z\. This is almost certainly not the intended behavior. Please ensure that the adapter name, 'z6', does not conflict with any of the targeted modules.rzTarget modules zL not found in the base model. Please check the target modules and try again.)rrrrrrr.rrrrrr>rsetattrrr difference)r{rrr=strict_module_check wrapper_clsactivate_adapterwrapper_kwargsrtrainable_modules found_modulesrIr!key_listtarget_module_foundrrrrr not_founds ` rN_set_trainablers28*  EM"'"5"5u"5"MN"M"MHN!!Z\!ZZ  7WX]_b7c 4FK+~66!=k]J^_cdo_p^qr$~%[]+{33""<B>B'' (B(BSa'bFK00 l=n=""6#8#8"X(PP #**<*W**2n*M Z8!((4   k *MPL!,,];I=i[(t u   aOsGcUR5H[n[U[5(dMURU5nXCR;aUR XBS9 MKUR /US9 M] g)z8Call set_adapter on the AuxiliaryTrainingWrapper modulesrN)rrrr8rr>)r{rr=r_rs rN _set_adapterrGsd--/ f6 7 7"(":":<"H #&6&66""#6"V""2n"E"rMc>Un[US5(aUR5nOUnSU;aUSnURcS[US5(a URnO/SU;aUSnO#SU;aUSnOSU;aUSnO [ S5eX0lUR c5SU;aUSnO#SU;aUSnOS U;aUS nO [ S 5eX@lUR cAS U;aUS nO/S U;aUS nO#S U;aUS nOSU;aUSnO [ S5eXPlURS;aLSU;aFUSnURSS5bUSnOUR UR -nXv-UlX`l[USS5cUR Ul U$)Nto_dict text_confignum_hidden_layers num_layersn_layerz,Please specify `num_layers` in `peft_config` hidden_sizen_embdd_modelz+Please specify `token_dim` in `peft_config`num_attention_headsn_head num_headsencoder_attention_headsz5Please specify `num_attention_heads` in `peft_config`> CARTRIDGE PREFIX_TUNINGnum_key_value_headshead_dimencoder_hidden_size) rIrrrr token_dimr" peft_typerrer*) peft_config model_configorig_model_configrr+r"r(r)s rN_prepare_prompt_learning_configr0Vs$|Y''#++- #  $#M2 % $&9 : :*< *='{148@*5*?*? ' rMc[5n[US5(dU$[(a UR$U/n[ U5S:aUR S5nUR RU;aV[U[5(a%URbU[UR5-nU[UR55- n[ U5S:aMU$)z Get the modules of the model that should not be split when using device_map. We iterate through the modules to get the underlying `_no_split_modules`. Returns: `List[str]`: List of modules that should not be split _no_split_modulesrr) rrIrr2rwr-rlrmrrrschildren)r{r2modules_to_checkr_s rN_get_no_split_modulesr5s#&% 5- . .    &&&w  ! #!%%b)    $ $,= =&/22++7(9C@X@XS 9n[,R."UU S 9n[,R."X/U/S 9nU$)Nget_module_class_from_namer)r7) _or_policylambda_auto_wrap_policytransformer_auto_wrap_policyr)CartridgeEncoder PrefixEncoderPromptEmbedding PromptEncoder,FSDP_TRANSFORMER_CLS_TO_WRAPz@Could not find the transformer layer class to wrap in the model.c[[UR555S:Xa*[USS5bURR (agg)NrrTF)rwrsnamed_childrenrerrg)r_s rNlambda_policy_fn/fsdp_auto_wrap_policy..lambda_policy_fns? V**,- .! 3$/; ++rM) lambda_fn)transformer_layer_cls)policies)rIr r7accelerate.utils.dataclassestorch.distributed.fsdp.wrapr8r9r:tunersr;r<r=r>rr5osenvironrrrw Exceptionr functoolspartial)r{r7r8r9r:r;r<r=r>%default_transformer_cls_names_to_wraptransformer_cls_names_to_wraptransformer_cls_to_wrap layer_classtransformer_clsrC lambda_policytransformer_wrap_policyauto_wrap_policys rNfsdp_auto_wrap_policyrXs-/KLL%C%^%^"KmmXX,/HH5J55Q,R)$&JJNN&(M% eCj" 0 _4 { q 4UH  "^_ _ # ' ' 85%%&=ZM'//$5 !((Ne>fg rMcU(dU$[U[RR5(a)[RRUR5$UR$r)rrGrrT)rfan_in_fan_outs rN transposer\sC  &%((,,--xx!!&((++ 88OrMcURU5(a-[U5[U5:aURSU-5$gg)z Helper function to match module names target_key and key. Makes sure that either the key is exactly the target_key or the target_key is a submodule of key rTF)rrw)rIr s rN_is_valid_matchr^s=  ||J s8c*o %<<j 01 1 rMcnUcUc [S5eUbURSnU$URSnU$)zfGet the batch size based on either input_ids or input_embeds Raises an ValueError if both are None. z5You have to provide either input_ids or inputs_embedsr)rr)r inputs_embeds batch_sizes rN_get_batch_sizerbsO  5PQQ__Q'  #((+ rMc[US5(aB[URS5(a'[USS5U:XaURR$g)z@ Get the quantization config of the related quantization method configquantization_configrSN)rIrdrere)r{methods rNget_quantization_configrgsH x  ELL"7 8 8 U14 8F B||/// rMc tUcg[5(dgSSKJn SSKJn SSKJn URnURnURn[US5(a UROSnURn [US5(a UROSn U"UUUU UUURU URS S 9 n U $) zK Get the right GPTQQuantLinear class based on the quantization config file Nr)BACKEND)METHOD)hf_select_quant_linear_v2checkpoint_formatrTmetaF) bits group_sizedesc_actsym device_mapformat quant_methodrmbackendpack)r gptqmodelrigptqmodel.quantizationrjgptqmodel.utils.importerrkrprornrIrlrqrmGPTQAUTO_TRAINABLE) gptq_quantization_configrrrirjrkrprornrlrqrm QuantLinears rNget_gptqmodel_quant_linearr~s ' ! # #!-B'00H)44J # ( (D +-@ A A !22  # & &C,34Lf,U,U # ( ([_D+   [[ &&  K rMcURRS:Xa/[5(a SSKnURR U5nO [ U5nURU[U54$)ad Unique identifier to a tensor storage. Multiple different tensors can share the same underlying storage. For example, "meta" tensors all share the same storage, and thus their identifier will all be equal. This identifier is guaranteed to be unique and constant for this tensor's storage during its lifetime. Two tensor storages with non-overlapping lifetimes may have the same id. This method is the exact same copy of https://github.com/huggingface/transformers/blob/main/src/transformers/pytorch_utils.py#L282C1-L300C58 but we added it here manually to avoid import issue with old versions of transformers. xlarN)devicerr torch_xla_XLAC_xla_get_tensor_idrr)tensorr unique_ids rNid_tensor_storager;sX}}U"'='?'? OO66v> ' ==)\&%9 99rMcUR5HPnUR(dURU5UlM,UR[R 5UlMR g)a Cast all non-trainable parameters of the model to the given `dtype`. The `dtype` can be `torch.float16` or `torch.bfloat16` as per the mixed-precision training you are performing. The trainable parameters are cast to full precision. This is meant to reduce the GPU memory usage when using PEFT methods by using half-precision dtype for non-trainable parameters. Having the trainable parameters in full-precision preserves training stability when using automatic mixed-precision training. Args: model (`torch.nn.Module`): The model to cast the non-trainable parameters of. dtype (`torch.dtype`): The dtype to cast the non-trainable parameters to. The `dtype` can be `torch.float16` or `torch.bfloat16` as per the mixed-precision training you are performing. N)rhrgrornrGrp)r{rirs rNcast_mixed_precision_paramsrTsA   TT%[AFTT%--(AF  rMcZUR5nUS;agUS;ag[SU35e)z Converts a string representation of truth to `True` (1) or `False` (0). True values are `y`, `yes`, `t`, `true`, `on`, and `1`; False value are `n`, `no`, `f`, `false`, `off`, and `0`; )yyesttrueon1r)rnoffalseoff0rzinvalid truth value )lowerr)values rN str_to_boolrjs: KKME 44 7 7/w788rMc Sn[[RRSS55(aU$[ X40UD6nU$![ [ 4a U$[a+n[R"SUSUSUS35 SnAU$SnAff=f)zCheck if a file exists on HF Hub, if check was not successful returns None instead of erroring. Respect offline mode if set. NHF_HUB_OFFLINErz7Unable to fetch remote file due to the following error z- - silently ignoring the lookup for the file z in r) rrKrLrrrrrMrxry)repo_idfilenamerexistses rNcheck_file_exists_on_hf_hubrzs "F2::>>"2C899   W9&9 M 1 2   M   EaSI%Jd7)1 6  M  s AB  B  BB c.[R"X5$)zBacking function for `target_modules` config parameter. Having this as its own function ensures that target key matching can be implemented in the same way everywhere. )re fullmatch)target_patternrIs rNmatch_target_against_keyrs << ,,rMcdUH)n[R"SUS3U5nU(dM'Us $ U$)z1Match a substring of key_to_match in pattern keysz(.*\.)?(z)$)rmatch) pattern_keys key_to_matchrIrs rNget_pattern_keyrs8HSE,l;   rMc x^^^[USS5b"[UUUR[USS5US9 [USS5bG[UR5RU5n[UUUR[USS5UUS9 [USS5Gb[ UR [5(a UR mO[US5nXaR 0m[USS5nUb THnX;dM [SUS US 35e URS S 5n [US S 5n S n [5n /n [T5S:GaYU (GaQ[[U55nUR5m[U4SjUR555nTHsnUR!U5(aU R#U5 M,UHAnUR!U5(dXR%S5;dM/U R#U5 Mq Mu [U 5S:aTU Sm['UU4SjU SS55(+n U (a@U (a9SR)U Vs/sH nUSTU3PM sn5n[SUS35eU (d[U 5U-n TR+5H,unnX;aM [UUURU/S[,UUS9 M. U (dU (a[.R0"S5 U =(a5 [ UR5[,5=(a U =(d U (+nU(ai[U5nUR5R2n[UUURUS[,UR4UUR5R2S9 gggs snf)aHandle the resolution of additional trainable modules (also called AuxiliaryTrainingWrapper) by checking the config if such modules are requested and adding them to the model. Currently trainable tokens and modules to save are considered additional trainable modules. If `activate_adapter` is set to `False`, the adapter won't be activated. This is typically the case when `model.add_adapter` or `model.load_adapter` are being called. rtN)r=rrmodules_to_tie)r=rrrvr embed_tokenszbThe embedding layer is already marked to be trained fully, either specify `modules_to_save=[..., "z ", ...]` or `trainable_tokens={'z': x}` but not both.tie_word_embeddingsFensure_weight_tyingrc3:># UHupUTLdM Uv M g7frrL)r*rmembedding_modules rNr+3set_additional_trainable_modules..s!_0EN^I^!!0Es  rrrc34># UH nTUT:Hv M g7frrL)r*rI first_indices target_layerss rNr+rs*nZmSV=+=+NZmsr z: zkCannot ensure weight tying when different token indices are specified for tied layers. Conflicting layers: zS. Please use the same indices for all tied layers or set ensure_weight_tying=False.T)r=rr rrrzensure_weight_tying=True but the model does not have tied weights (tie_word_embeddings=False). Weight tying will not be applied for trainable_token_indices.)r=rr rrr)rerr=rqrtrrdictrrrrrw%_get_module_names_tied_with_embeddingnextrrrrallrrErrxryrr)r{r-r.rrrvrGrttarget_layer_name weights_tiedrindices_mismatchlayers_to_skiptied_layer_keystied_module_namesembedding_namerItied_layers_inforshould_apply_tyingtied_weights_module_namesrrrrs @@@rN set_additional_trainable_modulesrs{-t4@  &55 .?F-  {,d3?e88:JJLY   &55 .>E-#  {5t<H k994 @ @'??M3E>JJ')L)LMM!+/@$G  &%2!$7$33D2EF00A/BBWY&3$''(=uE %k3H%P! }  !l #$I%$P Q  %99; !!_0C0C0E!__N&3!"**+<==#**+<=#4K"++,=>>BSWhWhilWmBm'../@A$5&3?#q( -oa.@ A '**nZijkjlZm*n'n#n $(;'+yy\k1l\kUXSEM#0C0C0E , } 2 *99/0$(2+!1  1F  3 MMm   >55579OP >$<,<(<  )Ne(T %!668FFM *996$(2+99,G"779GG  CIp2ms$N7c f[R"[R5[R"S5:nU(aSSKJn O [ S5e[XRU5n [U S5(aU R5OSn [U SS5n U cU b [U SS5n U cP[USS5n [USU 5n U "UR[R"XV4URS 9UUUU US 9nU$U "UUUR5URUUURUUS 9 nU$) Nz4.53.1r)create_masks_for_generatezDYour transformers version is too old, please upgrade it to >= 4.53.1 get_decoder5_prepare_4d_causal_attention_mask_with_cache_positiontoken_type_idsr)ri)rd input_embedsattention_maskcache_positionpast_key_valuesr position_ids)sequence_length target_lengthrirrardrr)rparse transformers __version__transformers.masking_utilsr ImportErrorrebase_model_prefixrIrrdrGemptyriget_max_cache_shape)r{ model_inputrrrrarrtransformers_ge_4_53_1r base_modeldecodercausal_mask_creation_functionrs rNcreate_attention_maskr:s=%]]<+C+CD V^H__H`aa 7 7?J*1*m*L*Lj$$&RVG$+J8oqu$v!$,1D(/9prv(w%%, .>E(/7RTm(n%6<<j%B%++V))+)%  , 7 +)==?++)!<<+%   rMc ^ ^/n[US5(aUR5n[US5(a URn[US5(d/$[R"S5n[ UR [5(Ga[US5(d [S5e0nUR5H{upE[USS5nU(aU(dURU5 M3U(dM<URUR5VVs0sHupxUSU3USU3_M snn5 M} [UR5R55n UR!S S 9V V s/sHupX;dM U PM sn n m T V s0sHoR#S U 5U _M sn mTR"S0T V s0sHoR%S S 5U _M sn D6 UR'U U4S jUR555 O(UR bUR'UR 5 [)UV s1sHoR+S5SiM sn 5$s snnfs sn n fs sn fs sn fs sn f)aG Get the list of the fully qualified names of the modules that are tied to the input embeddings. In case of a source-target-mapping `_tied_weights_keys`, it will attempt to identify the input embedding weights from the mapping and return the list of tied modules accordingly. This gives a unified interface to both transformers v4 tied weights and v5 mapped tied weights. For example: For models which have `embed_tokens` and `lm_head` as the tied keys, this function will return [`lm_head`]. The PEFT model is assumed to be transparent: returned names will be relative to the base model, so even though `model.base_model.lm_head` is tied, the returned name is `lm_head` since such attributes are forwarded to the base model anyway. Non-transformer models have to provide a `_tied_weights_keys` attribute for this function to work. Note that this function will not check if weight tying is disabled by the model's config. There can be the case that the weight tying definition is present but the tying is disabled via `model_config.tie_word_embeddings=False`. You have to check that yourself. get_base_modeltuner_layer_cls_tied_weights_keysz[^.]+\.base_layer\.rqzThe supplied model implements `_tied_weights_keys` as a dict but doesn't implement 'get_input_embeddings' so we can't determine which weights are tied to embeddings.NrFrz base_layer.c3x># UH/upTRX"5T;dMTRX5v M1 g7fr)r)r*rv candidatespeft_reverse_mappings rNr+8_get_module_names_tied_with_embedding..s> 1#''-; + $ $Q * *1s::rrL)rIrr{rcompilerrrrrrerrErrqrhrfsubreplaceextendsorted rpartition)r{ tied_weightsbase_layer_patterntied_weights_keys module_namer_module_tied_weights_keysrrinput_embedding_paramsrrrrrs @@rNrrosZ"!Lu&''$$&u'((  5. / / $:;%**D11u455e  -/#(#6#6#8 K'.v7KT'R $' !(()AB))!((LdLjLjLlmLlDA }AaS)k]!A3+??Llm $9"%U%?%?%A%L%L%N!O$)$:$:E$:$Rr$RDAVWVqa$Rr T^^S]4 6 6r4 @$ FS]^ ##eYc&dYcQU||M2'F'LYc&de )//1   ! ! -E445 |D|t??3'*|D EE=ns _'eEs$I! 2 I'I'I-I2 =I7)rYr)TN)rr\rintrr)rgrZrYrbr)FNT)r=rZr rZrz"Optional[AuxiliaryTrainingWrapper]rrZr`)rr_r=rZrYrbrX)rIrr r)rOptional[torch.Tensor]r`rrYr)r{rgrfr)rr\rYztuple[torch.device, int, int])rrrYr)rrrrrYzOptional[bool])rrrIr)rz Sequence[str]rrrYrre)rrZr[)u __future__rrrNrtrKrrxcollections.abcr contextlibroperatorrtypingrrr rrGrr accelerate.hooksr r accelerate.utilsr rhuggingface_hubrhuggingface_hub.errorsrr packagingrsafetensors.torchrrr import_utilsrrr constantsrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>rKrrr?__all__rOrrrrModulerrrrrrrrrrr0r5rXr\r^rbrgr~rrrrrrrrrrLrMrNrs#  $"'' 5H?'H7(dd%%%%%%%%%%P  ==''(GMM(,CC1$&M% R Ur*4&c"uxxc"L _03_0DpE5pEf ' 4$!&6:!V V  V 4 VVr F>B>'T   &R:2), 9 2-Qh2jMFrM