>:j& |SSKrSSKJr SSKrSrS\R S\S\R 4SjrS\R S\S\S\R 4S jr SS\R S\S \S S\S\R 4 S jjr SS\R S \S S\R 4Sjjr S\R S\R S\R 4Sjr S\ \R S\R S\R 4SjrS\ \R S\R S\S\R 4SjrSS\ \R S\R S\S\S S\R 4 SjjrS\ \R S\R S\S\R 4SjrSS\ \R S\R S\S\S S\R 4 Sjjrg)N)LiteralcURSUR5UR5- --nURU5nU$)a- Reshapes `weights` to match the shape of `task_tensors` by unsqeezing in the remaining dimenions. Args: task_tensors (`torch.Tensor`): The tensors that will be used to reshape `weights`. weights (`torch.Tensor`): The tensor to be reshaped. Returns: `torch.Tensor`: The reshaped tensor. ))shapedimview) task_tensorsweights new_shapes S/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/utils/merge_utils.pyreshape_weight_task_tensorsr s>  (8(8(:W[[](J KKIll9%G Ntensordensityreturnc0[R"U5RS5n[XR 5-5n[R "UR 5RS5USS9nSX$S'XRUR5-$)a> Prune the smallest values of the task tensors and retain the top-k values based on the specified fraction `density`. Args: tensor (`torch.Tensor`):The tensor to prune. density (`float`):The fraction of values to preserve. Should be in [0,1]. Returns: `torch.Tensor`: The tensor with the pruned weights. T)klargestr)torch zeros_likereshapeintnumeltopkabsr)rrmaskrtop_ks r magnitude_based_pruningr%sv   F # + +B /D Glln $%A JJvzz|++B/1d CEDqN LL. ..rrescalect[R"[R"XS95nX-nU(aXA- nU$)aa Prune random values based on the specified fraction `density`. Args: tensor (`torch.Tensor`):The tensor to prune. density (`float`):The fraction of values to preserve. Should be in [0,1]. rescale (`bool`):Whether to rescale the result to preserve the expected value of the original tensor. Returns: `torch.Tensor`: The pruned tensor. )input fill_value)r bernoulli full_like)rrr r pruned_tensors r random_pruningr'8s3 ??5??L MDMM%/ rmethod) magnituderandomcUS:a[R"SUS35 U$US:a[SU35eUS:Xa [X5$US:Xa [ XUS9$[S U35e) a Prune the values of task tensors based on the `method`. Args: tensor (`torch.Tensor`):The tensor to prune. density (`float`):The fraction of values to preserve. Should be in [0,1]. method (`str`):The method to use to prune. Should be one of ["magnitude", "random"]. rescale (`bool`):Whether to rescale the result to preserve the expected value of the original tensor. Returns: `torch.Tensor`: The pruned tensor. rz The density z= is greater than or equal to 1, no pruning will be performed.rzDensity should be >= 0, got r)r*)r zUnknown method )warningswarn ValueErrorrr')rrr(r s r pruner/Ksz!| WI-jkl 17yABB &v77 8 fw???6(344r)total frequencycUR5nUS:XaURSS9nO%US:XaURSS9nO[SUS35e[R"US:SS5nX$:H$) a> Get the mask of the majority sign across the task tensors. Task tensors are stacked on dimension 0. Args: tensor (`torch.Tensor`):The tensor to get the mask from. method (`str`):The method to use to get the mask. Should be one of ["total", "frequency"]. Returns: `torch.Tensor`: The majority sign mask. r0rrr1zUnimplemented mask method ""rr)signsum RuntimeErrorrwhere)rr(r5sign_magnitude majority_signs r calculate_majority_sign_maskr;gss ;;=D * ; a8BCCKK! 3Q;M   rr majority_sign_maskcrX-RSS9nURSS9nU[R"USS9- $)a Merge the task tensors using disjoint merge. Args: task_tensors (`torch.Tensor`):The task tensors to merge. majority_sign_mask (`torch.Tensor`):The mask of the majority sign across the task tensors. Returns: `torch.Tensor`: The merged tensor. rr3g?)min)r6rclamp)r r<mixed_task_tensorsnum_params_preserveds r disjoint_mergerBsF';@@Q@G-11a18  ,@c J JJrr cl[R"USS9n[X5nX-nURSS9nU$)z Merge the task tensors using `task arithmetic`. Args: task_tensors(`List[torch.Tensor]`):The task tensors to merge. weights (`torch.Tensor`):The weights of the task tensors. Returns: `torch.Tensor`: The merged tensor. rr3)rstackr r6)r r weighted_task_tensorsr@s r task_arithmeticrFsA;;|3L),@G(2.22q29 rc UVs/sH n[X2SS9PM nn[R"USS9n[X5nX-nUR SS9nU$s snf)a8 Merge the task tensors using `task arithmetic`. Args: task_tensors(`List[torch.Tensor]`):The task tensors to merge. weights (`torch.Tensor`):The weights of the task tensors. density (`float`): The fraction of values to preserve. Should be in [0,1]. Returns: `torch.Tensor`: The merged tensor. r)r(rr3r/rrDr r6r r rrrEr@s r magnitude_prunerKsdNZZ\6E&+>\LZ;;|3L),@G(2.22q29  [sAmajority_sign_methodc UVs/sH n[XBSS9PM nn[R"USS9n[XS9n[ X5nX-n[ Xe5nU$s snf)a Merge the task tensors using `ties`. Args: task_tensors(`List[torch.Tensor]`):The task tensors to merge. weights (`torch.Tensor`):The weights of the task tensors. density (`float`):The fraction of values to preserve. Should be in [0,1]. majority_sign_method (`str`): The method to use to get the majority sign mask. Should be one of ["total", "frequency"]. Returns: `torch.Tensor`: The merged tensor. r)rHrr3r/rrDr;r rBr r rrLrr<rEr@s r tiesrPsg(NZZ\6E&+>\LZ;;|3L5l`),@G(2'(=R [sAc UVs/sHn[X2SSS9PM nn[R"USS9n[X5nX-nUR SS9nU$s snf)a3 Merge the task tensors using `dare linear`. Args: task_tensors(`List[torch.Tensor]`):The task tensors to merge. weights (`torch.Tensor`):The weights of the task tensors. density (`float`):The fraction of values to preserve. Should be in [0,1]. Returns: `torch.Tensor`: The merged tensor. r*Tr(r rr3rIrJs r dare_linearrSshYeeXdfE&(DIXdLe;;|3L),@G(2.22q29  fsAc UVs/sHn[XBSSS9PM nn[R"USS9n[XS9n[ X5nX-n[ Xe5nU$s snf)a Merge the task tensors using `dare ties`. Args: task_tensors(`List[torch.Tensor]`):The task tensors to merge. weights (`torch.Tensor`):The weights of the task tensors. density (`float`):The fraction of values to preserve. Should be in [0,1]. majority_sign_method (`str`): The method to use to get the majority sign mask. Should be one of ["total", "frequency"]. Returns: `torch.Tensor`: The merged tensor. r*TrRrr3rHrNrOs r dare_tiesrUsk(YeeXdfE&(DIXdLe;;|3L5l`),@G(2'(=R fsA)F)r0)r,typingrrr Tensorfloatrboolr'r/r;rBlistrFrKrPrSrUrr r\s<   /ELL/5/U\\/&5<<%$5<<(ch5 LL5#(529:O2P5[_5 \\5:CJ! LL!")*>"?! \\!2 K K5<< KTYT`T` K $u||"4u||PUP\P\&$u||"4u||V[`e`l`l4;B u||$ \\""67   \\ @d5<<05<<RW\a\h\h4;B u||$ \\""67   \\ r