EiqfnSSKrSSKrSSKrSSKJs Jr SSKJrJr S/r"SS\R5r g)N)nnTensorMultiheadAttentionc^\rSrSr\R rS/rSS\S\S\ S\ S\ S \ S \S-S \S-S\ S S4U4S jjjr Sr \ S5r\R R"S5r\ S5rSS\S\S\S\S-S\ S\S-S\ S\ S \\\S-44SjjrSS\S\S\S\S-S\ S\S-S\ S\ S \\\S-44SjjrSrU=r$)r batch_firstN embed_dim num_headsdropoutbias add_bias_kv add_zero_attnkdimvdimreturnc >XS.n [T U]"UUUUUUUUU 4 0U D6 [R"URUR4SU0U D6Ul[R"UR UR4SU0U D6Ul[R"URUR4SU0U D6Ul [R"URUR4SU0U D6Ul [RRRR5Ul[RR R#5Ul[RR R#5Ul[RR R)5Ul[RR R)5Ul[RR R)5Ulg)N)devicedtyper )super__init__rLinearr linear_Qrlinear_Krlinear_Vout_projtorchao quantizedFloatFunctionalq_scaling_product quantization QuantStubquant_attn_outputquant_attn_output_weights DeQuantStub dequant_q dequant_k dequant_v)selfr r r r r rrrrrrfactory_kwargs __class__s g/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/ao/nn/quantizable/modules/activation.pyrMultiheadAttention.__init__=s%+;             NNDNN 15 9G   IIt~~ ,0 4B   IIt~~ ,0 4B   NNDNN 15 9G  "'!6!6!F!F!H"'!6!6!@!@!B).)>)>)H)H)J&..::<..::<..::<cg)NQuantizableMultiheadAttention)r)s r, _get_nameMultiheadAttention._get_namers.r.c  [U5URLa$[SURS[U535e[US5(d [S5eU"URUR UR URSLURSLURURURUR5 nURUlURUl URUlURR URlURR"URlUR$(GaURnSnXAR-nUR&XE2SS24nUb,[(R*R-X4UUR.5n[(R*R-XfR.5UR0lX2R0lURnUnXAR-nUR&XE2SS24nUb,[(R*R-X4UUR.5n[(R*R-XfR.5UR2lX2R2lURnUnUR&US2SS24nUb,[(R*R-X4SUR.5n[(R*R-XfR.5UR4lX2R4lGO[*R,"UR65UR0l[*R,"UR85UR2l[*R,"UR:5UR4lURc4SUR0lSUR2lSUR4lO[*R,"URSUR5UR0l[*R,"URURURS-5UR2l[*R,"URURS-S5UR4lUR=5 [(R>R@RCUSS9nU$) NzExpected type , got qconfigz$The float module must have 'qconfig'rT)inplace)"type _FLOAT_MODULEAssertionErrorhasattrr r r in_proj_biasbias_krrrrbias_vr6rweightr _qkv_same_embed_dimin_proj_weightrr Parameter requires_gradrrr q_proj_weight k_proj_weight v_proj_weightevalrr!prepare)clsotherobservedr _start_endr@s r, from_floatMultiheadAttention.from_floatus ;c// /  !2!2 36$u+G ui(( !GH H OO OO MM   t + \\ %    JJ JJ      ,,,, ==$)>>#8#8 !&!4!4  $ $ $%%DFOO+D))&+q.9Fxx))$d*;T=O=OP',xx'9'9&BVBV'WH   $%)   "%%DFOO+D))&+q.9Fxx))$d*;T=O=OP',xx'9'9&BVBV'WH   $%)   "%%DF))&'1*5Fxx))$w-9K9KL',xx'9'9&BVBV'WH   $%)   "')||E4G4G'HH   $')||E4G4G'HH   $')||E4G4G'HH   $!!))-!!&)-!!&)-!!&)+&&q5??;*!!&*,&&u%//A:MO*!!&*,&&!(;'>?*!!&  88((0040Hr.c URURURURURR 5SSLUR SLURURURUR5 nURUR:wa%[SURSUR35eUR b3[R"UR R55UlUR b3[R"UR R55UlUR"R 5up#[R"UR55UR"lUb%[R"U5UR"lURR 5upEUR5nUR(R 5upgUR5nUR*R 5upUR5nUR(GaSn XR-n XAR,X2SS24'UR.b+[1US:H5(d [S5eXQR.X&U n XR-n XaR,X2SS24'UR.b+[1US:H5(d [S5eXqR.X&U n XR,U S2SS24'UR.b,[1U S:H5(d [S5eXR.U S&U$[R"U5Ul[R"U5Ul[R"U5UlUR.c5SURlSUR(lSUR*lU$XQR.SUR&XqR.URURS -&XR.URS -S&U$) zUtility to convert the quantized MHA back to float. The motivation for this is that it is not trivial to convert the weights from the format that is used in the quantized version back to the float. Nz_qkv_same_embed_dim mismatch:  != rz Expected all bQ elements to be 0z Expected all bK elements to be 0z Expected all bV elements to be 0r7)r:r r r r _weight_biasr>rrrrrAr;rrC dequantizer?rr@r rrrBr=allrErFrG) r)fpwbwQbQwKbKwVbVrMrNs r,rUMultiheadAttention.dequantizesb   NN NN LL ]] ' ' )! ,D 8 [[ $    II II       ! !T%=%= = 01G1G0HTMeMeLfg  ;; " T[[%;%;%=>BI ;; " T[[%;%;%=>BI }}))+\\!,,.9  =!||ABKK ++- ]]_++- ]]_++- ]]_ ! ! !FLL(D02  fk1n -*27||()KLL/1,FLL(D02  fk1n -*27||()KLL/1,F,.  fgqj )*27||()KLL+-("  "||B/B !||B/B !||B/B &%) "%) "%) "  57BLL1EG  q0@B8:!1 45 r.c[S5e)NzdIt looks like you are trying to prepare an MHA module. Please, see the examples on quantizable MHAs.)NotImplementedError)rJrKs r, from_observed MultiheadAttention.from_observeds " 0  r.querykeyvaluekey_padding_mask need_weights attn_maskaverage_attn_weights is_causalc 2URUUUUUUUU5$)a Note:: Please, refer to :func:`~torch.nn.MultiheadAttention.forward` for more information Args: query, key, value: map a query and a set of key-value pairs to an output. See "Attention Is All You Need" for more details. key_padding_mask: if provided, specified padding elements in the key will be ignored by the attention. When given a binary mask and a value is True, the corresponding value on the attention layer will be ignored. need_weights: output attn_output_weights. attn_mask: 2D or 3D mask that prevents attention to certain positions. A 2D mask will be broadcasted for all the batches while a 3D mask allows to specify a different mask for the entries of each batch. Shape: - Inputs: - query: :math:`(L, N, E)` where L is the target sequence length, N is the batch size, E is the embedding dimension. :math:`(N, L, E)` if ``batch_first`` is ``True``. - key: :math:`(S, N, E)`, where S is the source sequence length, N is the batch size, E is the embedding dimension. :math:`(N, S, E)` if ``batch_first`` is ``True``. - value: :math:`(S, N, E)` where S is the source sequence length, N is the batch size, E is the embedding dimension. :math:`(N, S, E)` if ``batch_first`` is ``True``. - key_padding_mask: :math:`(N, S)` where N is the batch size, S is the source sequence length. If a BoolTensor is provided, the positions with the value of ``True`` will be ignored while the position with the value of ``False`` will be unchanged. - attn_mask: 2D mask :math:`(L, S)` where L is the target sequence length, S is the source sequence length. 3D mask :math:`(N*num_heads, L, S)` where N is the batch size, L is the target sequence length, S is the source sequence length. attn_mask ensure that position i is allowed to attend the unmasked positions. If a BoolTensor is provided, positions with ``True`` is not allowed to attend while ``False`` values will be unchanged. If a FloatTensor is provided, it will be added to the attention weight. - is_causal: If specified, applies a causal mask as attention mask. Mutually exclusive with providing attn_mask. Default: ``False``. - average_attn_weights: If true, indicates that the returned ``attn_weights`` should be averaged across heads. Otherwise, ``attn_weights`` are provided separately per head. Note that this flag only has an effect when ``need_weights=True.``. Default: True (i.e. average weights across heads) - Outputs: - attn_output: :math:`(L, N, E)` where L is the target sequence length, N is the batch size, E is the embedding dimension. :math:`(N, L, E)` if ``batch_first`` is ``True``. - attn_output_weights: If ``average_attn_weights=True``, returns attention weights averaged across heads of shape :math:`(N, L, S)`, where N is the batch size, L is the target sequence length, S is the source sequence length. If ``average_attn_weights=False``, returns attention weights per head of shape :math:`(N, num_heads, L, S)`. ) _forward_impl) r)rerfrgrhrirjrkrls r,forwardMultiheadAttention.forward)s0r!!         r.c Sn Sn UbU(a [S5eU(a [S5eUR(aSXU45upnUR5upn URU :wa[SURSU 35eURS5URS5:wd$URS5URS5:wa-[SUR5S UR535eURUR-nXR-UR:wa [S 5e[ U5S -nUR U5nURU5nURU5nURRUU5nUGbUR[R:Xa4[R"S S S9 UR![R"5nUR%5(d6UR[R":wa[SUR35eUR'5S:Xa[UR)S5n[+UR55SURS5URS5/:wa [-S5eOUR'5S :XaV[+UR55XR-URS5URS5/:wa [-S5eO[-SUR'5S35eUbRUR[R:Xa4[R"SS S9 UR![R"5nUR.bUR0bU cU cUR.nUc [S5eUR0nUc [S5e[R2"UUR5SU S5/5n[R2"UUR5SU S5/5nUb[6R8"US5nUb[6R8"US5nOMU b [S5eU b [S5eO0UR.b [S5eUR0b [S5eUR;5R=XUR-U5R?SS5nUb=UR;5R=SXR-U5R?SS5nUb=UR;5R=SXR-U5R?SS5nU bU RS5XR-:wa,[SXR-SU RS535eU RS5U:wa [S USU RS535eU nU bU RS5XR-:wa,[S!XR-SU RS535eU RS5U:wa [S"USU RS535eU nURS5nUbjURS5U :wa [S#U SURS535eURS5U:wa [S$USURS535eUR@(Ga}US- n[RB"URS5S4UR5SS-5nURD(a?[RF"UURI5URK5UR5n[R2"UU/SS%9n[RB"URS5S4UR5SS-5nURD(a?[RF"UURI5URK5UR5n[R2"UU/SS%9nUb[6R8"US5nUb[6R8"US5nURMU5nUROU5nURQU5n[RR"UUR?SS55nXR-U U/n[+UR55U:wa([S&US[+UR5535eUb?UR[R":XaURUU[ S'55 OUU- nUbuUR=XRU U5nURWUR)S5R)S5[ S'55nUR=XR-U U5n[6RX"USS%9n[6RZ"UURZUR\S(9n[RR"UU5nXR-X/n[+UR55U:wa([S)US[+UR5535eUR(aUR=XUR5nO:UR?SS5R;5R=XUR5nUR_U5nURaU5nURcU5nU(a7UR=XRU U5nU(aUReSS%9nUU4$US4$)*Nz#Only allow causal mask or attn_maskz*causal mask not supported by AO MHA modulec3D# UHoRSS5v M g7f)rrRN) transpose).0xs r, 3MultiheadAttention._forward_impl..s Pr?catrepeatFpad contiguousviewrsrzeros is_quantizedquantize_per_tensorq_scale q_zero_pointr&r'r(bmm masked_fill_ masked_fillsoftmaxr r}r#rr$mean)r)rerfrgrhrirjrkrlstatic_kstatic_vtgt_lenbszembed_dim_to_checkhead_dimscalingqkvr>r?src_lenk_zerosv_zerosattn_output_weights expected_size attn_outputexpected_output_sizes r,rn MultiheadAttention._forward_implmsp   Y !FG G  !MN N    PU>/ / &t~~&6d;M:NO  88A;%**Q- '388A;%**Q-+G :388:,oV[V`V`VbUcd >>T^^3 nn $ 6 !KL L/T) MM%  MM#  MM%   " " - -a 9  %++- /  &LL4 ..00Y__ 5R$QR[RaRaQbc}}!#%//2  ()aA -LL&'UVVMA% ()..(JJqMHHQK. ''UVV #,Y]]_,==NO  ',<,B,Bekk,Q MM+  0225::>  ;; "t{{'>H$4>()BCC>()BCCIIq&--3":;<IIq&--3":;<( !i 8I#/'(uu-=v'F$'()NOO'()PQQ({{&$G{{&$G LLN  t~~)=x H R RSTVW X = ##Bnn(}oN/44678:   %**,#00E&MJ#y0#  '"5":":^^Wg# #6"A"A **1-77:f # #6":":nn$gw#  ii(;Dii 4<<$-- ii 3Q7 #nn 4gH    " #'; ; 67K6LMK,,./02    %**3HK%%a+gDNN3 ,,[9 mmK0 "<<=PQ "5":":^^Wg# $&9&>&>1&>&E# 33 3$ $r.) r'r&r(rrrrr r#r$) gTFFNNFNN)NTNTF)__name__ __module__ __qualname____firstlineno__rrr: __constants__intrrrr2 classmethodrOrjitunusedrUrcrtuplerorn__static_attributes__ __classcell__)r+s@r,rr s#))M*V#OM !#!3=3=3= 3=  3=  3=3=Dj3=Dj3=3= 3=3=j/NN` YYUUn  +/!#'%)B B B  B !4- B  B D=B #B B  vv}$ %B R+/!#'%)B%B%B% B% !4- B%  B%D=B%#B%B% vv}$ %B%B%r.) rr torch.jittorch.nn.functionalr functionalrr__all__rr1r.r,rs5  b %..b %r.