>:jJSSKr"SS\RR5rg)Nct^\rSrSrSrU4SjrS\R4SjrS\RSS4S jr S r U=r $) PrefixEncodera The `torch.nn` model to encode the prefix. Args: config ([`PrefixTuningConfig`]): The configuration of the prefix encoder. Example: ```py >>> from peft import PrefixEncoder, PrefixTuningConfig >>> config = PrefixTuningConfig( ... peft_type="PREFIX_TUNING", ... task_type="SEQ_2_SEQ_LM", ... num_virtual_tokens=20, ... token_dim=768, ... num_transformer_submodules=1, ... num_attention_heads=12, ... num_layers=12, ... encoder_hidden_size=768, ... ) >>> prefix_encoder = PrefixEncoder(config) ``` **Attributes**: - **embedding** (`torch.nn.Embedding`) -- The embedding layer of the prefix encoder. - **transform** (`torch.nn.Sequential`) -- The two-layer MLP to transform the prefix embeddings if `prefix_projection` is `True`. - **prefix_projection** (`bool`) -- Whether to project the prefix embeddings. Input shape: (`batch_size`, `num_virtual_tokens`) Output shape: (`batch_size`, `num_virtual_tokens`, `2*layers*hidden`) c x>[TU]5 URUlURnURnUR nUR nURnUR(GaTUR(GdB[RRXR5Ul [RR[RRX$5[RR5[RRXCS-U-55UlUS:Xa[RR"R%UR SR&R(5 [RR"R%UR SR*R(5 gg[RRXSS-U-5Ul US:XaH[RR"R%URR&R(5 gg)Nzero)super__init__prefix_projection token_dim num_layersencoder_hidden_sizenum_virtual_tokens init_weightsinference_modetorchnn Embedding embedding SequentialLinearTanh transforminitzeros_weightdatabias)selfconfigr rrrr __class__s \/home/mande/repo/quber/.venv/lib/python3.13/site-packages/peft/tuners/prefix_tuning/model.pyr PrefixEncoder.__init__8s !'!9!9$$ && $88#66**  ! ! !&*?*?*?"XX//0BNDN"XX00 ?  3!^i5OPDN v% $$T^^B%7%>%>%C%CD $$T^^B%7%<%<%A%AB&#XX//0BQRNU^D^_DNv% $$T^^%:%:%?%?@&prefixcUR(a$URU5nURU5nU$URU5nU$)N)r rr)r r& prefix_tokenspast_key_valuess r#forwardPrefixEncoder.forwardQsE  ! ! NN62M"nn];O#nnV4Or%prompt_embeddingsreturnNc2UR(a [S5eURURRR:waL[S[ URRR5S[ UR5S35e[ R"5 URRRURURRR55 SSS5 g!,(df  g=f)a Load the flattened prompt embeddings saved by PEFT (`prompt_embeddings`). For prefix tuning, this is only supported when `prefix_projection=False`, because in that case the learned parameters are the KV prefix itself (`embedding.weight` has shape `[num_virtual_tokens, num_layers*2*token_dim]`). If `prefix_projection=True`, the parameters are (virtual token embeddings + an MLP) and there is no general way to invert the projection to recover those parameters from a flattened KV prefix. zFCannot load flattened prompt embeddings when `prefix_projection=True`.z,Invalid `prompt_embeddings` shape. Expected z, got .N) r ValueErrorshaperrtuplerno_gradcopy_todevice)r r,s r#load_prompt_embeddings$PrefixEncoder.load_prompt_embeddingsYs  ! !ef f  " "dnn&;&;&A&A A>..4456fUCTCZCZ=[<\\]_ ]]_ NN ! ! ' '(9(<(rCs"" X\EHHOOX\r%