"EiOD"SSKJr SSKrSSKJr SSKJrJrJrJrJ r J r J r J r J r JrJrJrJrJrJr SS/r"SS\5rS S \S \ S \ S \S \S 3 -\lS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\S\SS4SjrS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\S\SS4Sjr\"\S9S"S\\S\\S\\S\\S\\S \S-S\S\S\S\S\S\S\S\S\SS4 S!jj5rg)#)castN)Tensor)_capturable_doc_default_to_fused_or_foreach_differentiable_doc_disable_dynamo_if_unsupported _foreach_doc!_get_capturable_supported_devices_get_scalar_dtype _get_value _maximize_doc _params_doc _to_scalar_use_grad_for_differentiable _view_as_real OptimizerParamsTAdamaxadamaxc^\rSrSrSSSSS.S\S\\-S\\\4S\S \S \S-S \S \S \SS4U4Sjjjjr U4Sjr Sr \ SSj5r SrU=r$)rNF)maximizedifferentiable capturableparamslrbetaseps weight_decayforeachrrrreturnc >[U[5(aUR5S:wa [S5eSU::d[SU35eSU::d[SU35eSUSs=::aS:dO [SUS35eSUSs=::aS:dO [S US35eSU::d[S U35eUUUUUUUU S .n [T U]X5 g) NrzTensor lr must be 1-elementzInvalid learning rate: zInvalid epsilon value: r?z#Invalid beta parameter at index 0: z#Invalid beta parameter at index 1: zInvalid weight_decay value: )rrrr r!rrr) isinstancernumel ValueErrorsuper__init__) selfrrrrr r!rrrdefaults __class__s O/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/optim/adamax.pyr*Adamax.__init__s b& ! !bhhjAo:; ;by6rd;< <cz6se<= =eAh$$B58*MN NeAh$$B58*MN Nl";L>JK K( ,$   *cP>[TU]U5 URGHnURSS5 URSS5 URSS5 URSS5 USHnURR U/5n[ U5S:wdM0[R"US5(aMP[US5nUS(a([R"U[5URS 9O[R"U[5S 9US'M GM g) Nr!rFrrrrstepdtypedevicer4) r) __setstate__ param_groups setdefaultstategetlentorch is_tensorfloattensorr r5)r+r:grouppp_statestep_valr-s r.r7Adamax.__setstate__Ds U#&&E   Y -   Z /   -u 5   \5 18_**..B/w<1$U__WV_-M-M$WV_5H !. $,=,?#\\(:K:MN FO % 'r0c SnUSGHvnURcMU[R"U5-nURU5 URR(a [ S5eURUR5 UR Un [U 5S:XaUS(a([R"S[5URS9O[R"S[5S 9U S '[R"U[RS 9U S '[R"U[RS 9U S 'URU S 5 URU S 5 URU S 5 GMy U$)NFrz(Adamax does not support sparse gradientsrrr3r$r6r2) memory_formatexp_avgexp_inf)gradr= is_complexappend is_sparse RuntimeErrorr:r<zerosr r5r@ zeros_likepreserve_format) r+rAparams_with_gradgradsexp_avgsexp_infs state_steps has_complexrBr:s r. _init_groupAdamax._init_groupWsJ xAvv~ 5++A. .K  # #A &vv"#MNN LL JJqME5zQ\*KK*;*=ahhOc1B1DEf $)#3#3U%:%:$i $)#3#3U%:%:$i  OOE), - OOE), -   uV} -7!:r0c~UR5 SnUb%[R"5 U"5nSSS5 URHan/n/n/n/n/nUSupUSn USn USn USnUSnUSnUS nUR X4XVXx5n[ UUUUUU U U U U UUUUUS 9 Mc U$!,(df  N=f) zPerforms a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrrrr r!rrr) rbeta1beta2rr r!rrrrX)'_accelerator_graph_capture_health_checkr= enable_gradr8rYr)r+closurelossrArSrTrUrVrWr\r]rrr r!rrrrXs r.r2 Adamax.stepzs 446  ""$y%&&E-/ "$E%'H%'H(*K >LE,CtB 0LI&GZ(H"#34N|,J**(K  )!-%' )'L S%$s B.. B<rG)gMb`?)g?g+?g:0yE>rNN)__name__ __module__ __qualname____firstlineno__rr?rtupleboolr*r7rYrr2__static_attributes__ __classcell__)r-s@r.rrs"%1#$+$ $+$+ FN$+UE\" $+  $+  $+$+$+$+$+ $+$+L&!F"4"4r0aImplements Adamax algorithm (a variant of Adam based on infinity norm). .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \gamma \text{ (lr)}, \beta_1, \beta_2 \text{ (betas)},\theta_0 \text{ (params)},f(\theta) \text{ (objective)}, \: \lambda \text{ (weight decay)}, \\ &\hspace{13mm} \epsilon \text{ (epsilon)} \\ &\textbf{initialize} : m_0 \leftarrow 0 \text{ ( first moment)}, u_0 \leftarrow 0 \text{ ( infinity norm)} \\[-1.ex] &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm}if \: \lambda \neq 0 \\ &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1} \\ &\hspace{5mm}m_t \leftarrow \beta_1 m_{t-1} + (1 - \beta_1) g_t \\ &\hspace{5mm}u_t \leftarrow \mathrm{max}(\beta_2 u_{t-1}, |g_{t}|+\epsilon) \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1} - \frac{\gamma m_t}{(1-\beta^t_1) u_t} \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to `Adam: A Method for Stochastic Optimization`_. z Args: a lr (float, Tensor, optional): learning rate (default: 2e-3) betas (Tuple[float, float], optional): coefficients used for computing running averages of gradient and its square eps (float, optional): term added to the denominator to improve numerical stability (default: 1e-8) weight_decay (float, optional): weight decay (L2 penalty) (default: 0) z zd .. _Adam\: A Method for Stochastic Optimization: https://arxiv.org/abs/1412.6980 rrTrUrVrWrr\r]rr rrrrXr"c [RR5(d [U5n[ U5GHupXnU (dUOU*nX.nX>nXNn[R R 5(dhU (aa[5nURRURR:XaURRU;d[SUS35eUS- nU S:waURXS9n[R"U5(aX[R"U5n[R"U5n[R"U5n[R"U5nURUSU- 5 U (dC[R"UR!U5UR#5R%U5US9 O[R&"UR!U5R)S5UR#5R%U5R+S5/S5nUR-[R."USSS95 U (a3UU-S- nUR1U5 UU-nUR3UU5 GM[SU[5U5-- nUU- nUR3UUU*S 9 GM g) NIIf capturable=True, params and state_steps must be on supported devices: .rralpha)outF)keepdim)value)r=jit is_scriptingr enumeratecompiler is_compilingr r5typeAssertionErroraddrL view_as_reallerp_maximummul_absadd_cat unsqueeze unsqueeze_copy_amaxdiv_addcdiv_r )rrTrUrVrWrr\r]rr rrrrXiparamrKrIrJstep_tcapturable_supported_devicesnorm_bufneg_bias_correctiondenombias_correctionclrs r._single_tensor_adamaxrsN" 99 ! ! # # ^f%x#t$++~~**,,+L+N ( !!V]]%7%77LL%%)EE$_`|_}}~ !  1 88E86D   E " "&&u-E%%d+D((1G((1G  dAI& MM U# $  yye$..q1488:??33G3R3RST3UVH MM%**Xq%@ A #(-!"3   $ $R (11E NN7E *%:f+="==O&C NN7GC4N 8s&r0c F^U (a [S5e[U5S:Xag[RR 5(dAU (a:[ SS9m[ U4Sj[XSS955(d[STS 35e[U5n[R"XX#U/5nUR5GHuunnnnnn[[[U5n[[[U5n[[[U5n[[[U5n[[[U5nU (a[UUUU5 U (a[R "U5n[RR 5(d>USR"(a*[R$"U[R&"S S S 9S S 9 O[R$"US5 U S:wa4U (a[R$"UUU S 9 O[R("UUU S 9n[R*"UUSU- 5 [R,"UU5 U (dU S:Xa[R."U5nO[R0"U5 [R$"UU5 [R2"UU5 U (aw[R4"UU5n[R6"US5 [R8"UU5 [R:"UU5n[R<"UUU5 GM|UVs/sHnSU[?U5-- PM nnUVs/sHn[?U5U- S-PM nn[R<"UUUU5 GM gs snfs snf)Nz#_foreach ops don't support autogradrF) supports_xlac3># UHTupURRURR:H=(a URRT;v MV g7frc)r5ry).0rBr2rs r. '_multi_tensor_adamax..NsO A HHMMT[[-- - > !== >@sAAT)strictrmrnr%cpu)r5ror) rzr<r=rwrxr allziprr"_group_tensors_by_device_and_dtypevaluesrlistrr _foreach_negis_cpu _foreach_add_r@ _foreach_add_foreach_lerp_ _foreach_mul_ _foreach_abs _foreach_abs__foreach_maximum_ _foreach_pow _foreach_sub_ _foreach_div_ _foreach_mul_foreach_addcdiv_r ) rrTrUrVrWrr\r]rr rrrrXgrouped_tensorsgrouped_params_grouped_grads_grouped_exp_avgs_grouped_exp_infs_grouped_state_steps__grouped_params grouped_gradsgrouped_exp_avgsgrouped_exp_infsgrouped_state_stepsbias_correctionsrr2bc step_sizers @r._multi_tensor_adamaxr2s?"BCC 6{a >> & & ( (Z'H( $ v4@   ![\x[yyz{  BBBB K8O  " " $   d6lO<T&\>: V .?@V .?@"4<1EF   /?AQ  !..}=M ~~**,,1DQ1G1N1N   #U\\#e%DC     3Q 7 1 ##M>V % 2 2!>! -}a%iH ,e4LA-!..}=M    . M3/  0-@ $11%9LM     0! 4    0" 5&&'79IJE  # #N4De L;N :M$EZ---:M  ?OO>N*R.2-3>NIO  # # 02BI C %z Ps <NN)single_tensor_fnr!c [RR5(d"[SU55(d [ S5eUc [ XSS9upU(a.[R R5(a [ S5eU(a*[R R5(d[nO[nU"UUUUUU U U U UUUU US9 g)zjFunctional API that performs adamax algorithm computation. See :class:`~torch.optim.Adamax` for details. c3V# UHn[U[R5v M! g7frc)r&r=r)rts r.radamax..s!5-8 1ell##[s')zPAPI has changed, `state_steps` argument must contain a list of singleton tensorsNF) use_fusedz6torch.jit.script not supported with foreach optimizers) rr\r]rr rrrXr) r=rwrxrrOrrtrurr)rrTrUrVrWr!rrrrXrr\r]rr rfuncs r.rrs4 >> & & ( (5-8522 ^  1 e 599))++STTuyy--//#$   !%r0)NFFFF)typingrr=r optimizerrrrr r r r r rrrrrrr__all__r__doc__rr?rirrrrGr0r.rs & X RYRl4       5+`M9 LM9 <M96lM96l M9 f M9 M9 M9 M9 M9M9M9M9M9M9  !M9`s Ls <s6ls6l s f s s s s ssssss  !sl 1FG  < L< <<6l<6l < f <D[<<<<< < < !<" #<$%<& '<H<r0