Eis.8SSKJr SSKr/SQr"SS\RR 5r"SS\RR5r"SS \RR5r"S S \RR5r "S S \RR5r "SS\RR5r "SS\RRRR5r"SS\RR5rg))warnN)ReLU6 HardswishELU LeakyReLUSigmoidSoftmaxMultiheadAttentionPReLUcP^\rSrSrSrSU4SjjrSrSr\SSj5r Sr U=r $) raApplies the element-wise function: :math:`\text{ReLU6}(x) = \min(\max(x_0, x), q(6))`, where :math:`x_0` is the zero_point, and :math:`q(6)` is the quantized representation of number 6. Args: inplace: can optionally do the operation in-place. Default: ``False`` Shape: - Input: :math:`(N, *)` where `*` means, any number of additional dimensions - Output: :math:`(N, *)`, same shape as the input .. image:: ../scripts/activation_images/ReLU6.png Examples:: >>> m = nn.quantized.ReLU6() >>> input = torch.randn(2) >>> # xdoctest: +SKIP >>> input = torch.quantize_per_tensor(input, 1.0, 0, dtype=torch.qint32) >>> output = m(input) c0>[TU]U5 XlgN)super__init__inplace)selfr __class__s e/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/ao/nn/quantized/modules/activation.pyrReLU6.__init__,s ! ch[RRRXR5$r)torchops quantizedrelu6rrinputs rforward ReLU6.forward0s!yy""(( ==rcg)NQuantizedReLU6rs r _get_nameReLU6._get_name3rc,[UR5$r)rr)moduse_precomputed_fake_quants r from_floatReLU6.from_float6sS[[!!r)rF) __name__ __module__ __qualname____firstlineno____doc__rrr% staticmethodr+__static_attributes__ __classcell__rs@rrrs+0> ""rrc`^\rSrSrSrS U4SjjrSrSr\S Sj5r \ S5r Sr U=r $) r;zThis is the quantized version of :class:`~torch.nn.Hardswish`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor c>X4S.n[TU]5 URS[R"U40UD65 URS[R"U40UD65 gN)devicedtypescale zero_pointrrregister_bufferrtensor)rr=r>r;r<factory_kwargsrs rrHardswish.__init__CsR$*;  Well5&KN&KL \5<< +Un+UVrc~[RRRXRUR 5$r)rrr hardswishr=r>rs rrHardswish.forwardKs'yy"",,UJJPPrcg)NQuantizedHardswishr#r$s rr%Hardswish._get_nameN#rcvURR5up#[[U5[ U55$r)activation_post_processcalculate_qparamsrfloatintr)r*r=r>s rr+Hardswish.from_floatQs.77IIKus:77rc8U"[U5[U55$r)rNrOclsr)r=r>s rfrom_referenceHardswish.from_referenceVs5<Z11rr#)NNr-r.r/r0r1r2rrr%r3r+ classmethodrUr4r5r6s@rrr;sAWQ$8822rrc`^\rSrSrSrS U4SjjrSrSr\S Sj5r \ S5r Sr U=r $) r[zThis is the quantized equivalent of :class:`~torch.nn.ELU`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor alpha: the alpha constant c<>[TU]U5 XlX lgr)rrr=r>)rr=r>alphars rr ELU.__init__ds  $rc[RRRRR XR URUR5$r) raonnr functionalelur=r>r\rs rr ELU.forwardis<xx{{$$//33 ::t   rcg)N QuantizedELUr#r$s rr% ELU._get_namensrcURR5up#[[U5[ U5UR 5$r)rLrMrrNrOr\rPs rr+ELU.from_floatqs477IIK5<Z#))<)?r-rWr6s@rrr[s?%  ====rrc ^\rSrSrSrSS\S\S\S\SS4 U4S jjjrS r S r \ SS j5r \ S 5r SrU=r$)r{aThis is the quantized equivalent of :class:`~torch.nn.LeakyReLU`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor negative_slope: Controls the angle of the negative slope. Default: 1e-2 Nr=r>negative_sloperreturnc>XVS.n[TU]X45 URS[R"U40UD65 URS[R"U40UD65 gr:r?) rr=r>rnrr;r<rBrs rrLeakyReLU.__init__sV%+; 1 Well5&KN&KL \5<< +Un+UVrc[RRRXRUR UR UR5$r)rrr leaky_relurnrr=r>rs rrLeakyReLU.forwards:yy""-- && djj$//  rcg)NQuantizedLeakyReLUr#r$s rr%LeakyReLU._get_namerJrcURR5up4U"[U5[U5URUR 5$r)rLrMrNrOrnr)rTr)r*r=r>s rr+LeakyReLU.from_floats<77IIK5<Z#2D2DckkRRrcdU"[U5[U5URUR5$r)rNrOrnrrSs rrULeakyReLU.from_references%5<Z#2D2DckkRRrr#)g{Gz?FNNr-)r.r/r0r1r2rNrOboolrrr%rXr+rUr4r5r6s@rrr{s!%WWW W  W WW  $SSSSrrcR^\rSrSrSrS\S\4U4SjjrSr\ S Sj5r Sr U=r $) rzThis is the quantized equivalent of :class:`~torch.nn.Sigmoid`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor output_scaleoutput_zero_pointc:>[TU]5 XlX lgr)rrrr)rrrrs rrSigmoid.__init__s (!2rc~[RRRXRUR 5$r)rrrsigmoidrrrs rrSigmoid.forwards0yy""** $$d&<&<  rcrURR5unnU"[U5[U55$r)rLrMrNrO)rTr)r*rrs rr+Sigmoid.from_floats9  ' ' 9 9 ;  5&,=(>??r)rrr-) r.r/r0r1r2rNrOrrrXr+r4r5r6s@rrrs63U3s3  @@rrc`^\rSrSrSrS U4SjjrSrSr\S Sj5r \ S5r Sr U=r $) r aThis is the quantized version of :class:`~torch.nn.Softmax`. Args: dim: A dimension along which Softmax will be computed (so every slice along dim will sum to 1). scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor cF>[TU]5 XlX lX0lgr)rrdimr=r>)rrr=r>rs rrSoftmax.__init__s  $rcURnUc;Sn[RRR SUR5U5n[R R RXURUR5$)Nsoftmax) rrr`ra_get_softmax_dimrrrr=r>)rrr stacklevels rrSoftmax.forwardsghh ;J((%%66599; Cyy""**5tzz4??SSrcg)NQuantizedSoftmaxr#r$s rr%Softmax._get_names!rcURR5up#[UR[ U5[ U55$r)rLrMr rrNrOrPs rr+Softmax.from_floats477IIKswwe c*o>>rcNU"UR[U5[U55$r)rrNrOrSs rrUSoftmax.from_references377E%L#j/::r)rr=r>)Nrkrr-rWr6s@rr r s@% T"??;;rr c\rSrSr\R R RRr Sr \ S5r \ S5r Srg)r cg)NQuantizedMultiheadAttentionr#r$s rr%MultiheadAttention._get_names,rc[S5e)NzpIt looks like you are trying to convert a non-observed MHA module. Please, see the examples on quantizable MHAs.)NotImplementedError)rTothers rr+MultiheadAttention.from_floats" 0  rc8[RRRUSSSSS9nXlUR beUR RS5n[R"USS9upE[R"X4U[R5n[USU5 URbeUR RS5n[R"WSS9upE[R"XdU[R5n[USU5 U? U?U$)NFT)mappingrremove_qconfigconvert_custom_config_dictbias_k) reduce_rangebias_v)rr_ quantizationconvertrr _parameterspop_choose_qparams_per_tensorquantize_per_tensorquint8setattrrin_proj_weight in_proj_bias)rTr convertedrsczprs r from_observed MultiheadAttention.from_observedsHH))11 '+ 2 "    '**..x8F55f5QFB..v2u||LF Ix 0    '**..x8F55"FB..v2u||LF Ix 0  $  "rr#N)r.r/r0r1rr_r` quantizabler _FLOAT_MODULEr%rXr+rr4r#rrr r sHHHKK++>>M-    rr c ^\rSrSrSrSS\S\S\SS4U4SjjjrS \RSS4S jr S \RS\R4S jr S r \ SSj5r\ S5rSrU=r$)r iaThis is the quantized equivalent of :class:`~torch.nn.PReLU`. Args: scale: quantization scale of the output tensor zero_point: quantization zero point of the output tensor num_parameters: number of parameters: 1, or the number of channels at input. Default: 1 rrnum_parametersroNc>[TU]5 X0lXlX l[ R "U[ RS9n[ R"USS[ RS9nURU5 g)N)r<rkr)r=r>r<) rrrr=r>rrandnrNrr set_weight)rrrrwqwrs rrPReLU.__init__&sZ ,! + KKekk :  & &q V rrcXlgr)weight)rrs rrPReLU.set_weight1s rrc[RRRXRUR UR 5$r)rrrprelurr=r>rs rr PReLU.forward4s2yy""(( ;; DOO  rcg)NQuantizedPReLUr#r$s rr%PReLU._get_name9r'rc$URR5up4U"[U5[U5UR5nUR R5nUR R 5nU"U5 UR[R:wa[SUR3SS9 UR5up[R"U[U5[U 5[R5n URU 5 U$Nz9PReLU's weight observer should have dtype quint8 but got )r) rLrMrNrOrrqconfigr<rrrrr) rTr)r*r=r>qprelufloat_wtobserverwt_scalewt_zpqweights rr+PReLU.from_float<s77IIKU5\3z?C4F4FG::##%;;%%' >>U\\ ) KHNNK[\ #446++ eHos5z5<<  '" rcU"[U5[U5UR5nURR5nURR5nU"U5 UR [ R:wa[SUR 3SS9 UR5upx[ R"U[U5[U5[ R5n URU 5 U$r) rNrOrrrr<rrrrMrr) rTr)r=r>rrrrrrs rrUPReLU.from_referenceOsU5\3z?C4F4FG::##%;;%%' >>U\\ ) KHNNK[\ #446++ eHos5z5<<  '" r)rr=rr>)r-)r.r/r0r1r2rNrOrrTensorrrr%rXr+rUr4r5r6s@rr r sRS ! 69 KN    ELLT U\\ ell  $rr )warningsrr__all__r`ReLUrrrrrr r_rr Moduler r#rrrs  %"EHHMM%"P2""2@=%((,,=@(S""(SV@ehh@6$;ehh$;N200CC2jBEHHOOBr