"EiA SSKJrJr SSKrSSKJr SSKJrJrJrJ r J r J r J r J r JrJrJrJrJrJr SS/r"SS\5rS S \S \ S \S \ S \S 3 -\lS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\SS4SjrS\\S\\S\\S\\S\\S\S\S\S\S\S\S\S\SS4Sjr\ "\S9S!S\\S\\S\\S\\S\\S\S\S-S\S\S\S\S\S\S\SS4S jj5rg)")AnycastN)Tensor)_capturable_doc_default_to_fused_or_foreach_differentiable_doc_disable_dynamo_if_unsupported _foreach_doc!_get_capturable_supported_devices_get_scalar_dtype _maximize_doc _params_doc _to_scalar_use_grad_for_differentiable _view_as_real OptimizerParamsTAdadeltaadadeltac^\rSrSrSSSSS.S\S\\-S\S\S \S \S-S \S \S \SS4U4SjjjjrU4Sjr S\ \ \ 4S\ \S\ \S\ \S\ \S\ \4 Sjr\SSj5rSrU=r$)rNF) capturablemaximizedifferentiableparamslrrhoeps weight_decayforeachrrrreturnc T>[U[5(aUR5S:wa [S5eSU::d[SU35eSUs=::aS::dO [SU35eSU::d[SU35eSU::d[SU35eUUUUUUUU S .n [T U]X5 g) NrzTensor lr must be 1-elementgzInvalid learning rate: ?zInvalid rho value: zInvalid epsilon value: zInvalid weight_decay value: )rrrr rrr!r) isinstancernumel ValueErrorsuper__init__) selfrrrrr r!rrrdefaults __class__s Q/home/mande/repo/quber/.venv/lib/python3.13/site-packages/torch/optim/adadelta.pyr)Adadelta.__init__s b& ! !bhhjAo:; ;by6rd;< <c S 23%89 9cz6se<= =l";L>JK K( $,   *cP>[TU]U5 URGHnURSS5 URSS5 URSS5 URSS5 USHnURR U/5n[ U5S:wdM0[R"US5(aMP[US5nUS(a([R"U[5URS 9O[R"U[5S 9US'M GM g) Nr!rFrrrrstepdtypedevicer3) r( __setstate__ param_groups setdefaultstategetlentorch is_tensorfloattensorr r4)r*r9grouppp_statestep_valr,s r-r6Adadelta.__setstate__As U#&&E   Y -   Z /   -u 5   \5 18_**..B/w<1$U__WV_-M-M$WV_5H !. $,=,?#\\(:K:MN FO % 'r/r@params_with_gradgrads square_avgs acc_deltas state_stepsc SnUSGHvnURcMU[R"U5-nURU5 URR(a [ S5eURUR5 UR Un [U 5S:XaUS(a([R"S[5URS9O[R"S[5S9U S '[R"U[RS 9U S '[R"U[RS 9U S 'URU S 5 URU S 5 URU S 5 GMy U$) NFrz*Adadelta does not support sparse gradientsrrr2r5r1) memory_format square_avg acc_delta) gradr< is_complexappend is_sparse RuntimeErrorr9r;zerosr r4 zeros_likepreserve_format) r*r@rErFrGrHrI has_complexrAr9s r- _init_groupAdadelta._init_groupTsN xAvv~ 5++A. .K  # #A &vv"#OPP LL JJqME5zQ\*KK*;*=ahhOR/@/BCf ',&6&6U%:%:'l#&+%5%5U%:%:&k"   u\2 3   eK0 1   uV} -9!<r/c~UR5 SnUb%[R"5 U"5nSSS5 URHan/n/n/n/n/nUSUSUSUSUSUSUSUS 4un n n n n nnnUR X4XVXx5n[ UUUUUU U U U U UUUUS 9 Mc U$!,(df  N=f) zPerform a single optimization step. Args: closure (Callable, optional): A closure that reevaluates the model and returns the loss. Nrrrr r!rrr) rrrr r!rrrrW)'_accelerator_graph_capture_health_checkr< enable_gradr7rXr)r*closurelossr@rErFrGrHrIrrrr r!rrrrWs r-r1 Adadelta.steps  446  ""$y%&&E-/ "$E(*K')J(*Kd e e n%i j!&'l#  **ZK  )!-%' ='^ e%$s B.. B<rK)r$g?gư>rNN)__name__ __module__ __qualname____firstlineno__rr>rboolr)r6dictstrrlistrXrr1__static_attributes__ __classcell__)r,s@r-rrs!#"+!$"+"+ FN"+ "+  "+  "+"+"+"+"+ "+"+H&)CH~)v,)F| ) &\ ) L )&\)V"="=r/aImplements Adadelta algorithm. .. math:: \begin{aligned} &\rule{110mm}{0.4pt} \\ &\textbf{input} : \gamma \text{ (lr)}, \: \theta_0 \text{ (params)}, \: f(\theta) \text{ (objective)}, \: \rho \text{ (decay)}, \: \lambda \text{ (weight decay)} \\ &\textbf{initialize} : v_0 \leftarrow 0 \: \text{ (square avg)}, \: u_0 \leftarrow 0 \: \text{ (accumulate variables)} \\[-1.ex] &\rule{110mm}{0.4pt} \\ &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \: \textbf{do} \\ &\hspace{5mm}g_t \leftarrow \nabla_{\theta} f_t (\theta_{t-1}) \\ &\hspace{5mm}if \: \lambda \neq 0 \\ &\hspace{10mm} g_t \leftarrow g_t + \lambda \theta_{t-1} \\ &\hspace{5mm} v_t \leftarrow v_{t-1} \rho + g^2_t (1 - \rho) \\ &\hspace{5mm}\Delta x_t \leftarrow \frac{\sqrt{u_{t-1} + \epsilon }}{ \sqrt{v_t + \epsilon} }g_t \hspace{21mm} \\ &\hspace{5mm} u_t \leftarrow u_{t-1} \rho + \Delta x^2_t (1 - \rho) \\ &\hspace{5mm}\theta_t \leftarrow \theta_{t-1} - \gamma \Delta x_t \\ &\rule{110mm}{0.4pt} \\[-1.ex] &\bf{return} \: \theta_t \\[-1.ex] &\rule{110mm}{0.4pt} \\[-1.ex] \end{aligned} For further details regarding the algorithm we refer to `ADADELTA: An Adaptive Learning Rate Method`_. z Args: ar lr (float, Tensor, optional): coefficient that scale delta before it is applied to the parameters (default: 1.0) rho (float, optional): coefficient used for computing a running average of squared gradients (default: 0.9). A higher value of `rho` will result in a slower average, which can be helpful for preventing oscillations in the learning process. eps (float, optional): term added to the denominator to improve numerical stability (default: 1e-6). weight_decay (float, optional): weight decay (L2 penalty) (default: 0) z zd .. _ADADELTA\: An Adaptive Learning Rate Method: https://arxiv.org/abs/1212.5701 rrFrGrHrIrrrr rrrrWr"c \^[RR5(dAU (a:[SS9m[ U4Sj[ XSS955(d[ STS35e[RR5(d [U5n[ XX#USS9GHupnnnUS- nU (dUOU*nUS :waURXS 9n[R"U 5(aB[R"U5n[R"U5n[R"U5nURU5RXSU- S 9 URU5R5nURU5R5nU (aUR!5nUR#U5RU5 URU5RUUSU- S 9 [R"U 5(a[R$"U5nU R'UU*S 9 GM g) NF supports_xlac3># UHTupURRURR:H=(a URRT;v MV g7fr`r4type.0rAr1capturable_supported_devicess r- *_single_tensor_adadelta.. O A HHMMT[[-- - > !== >@AATstrictIIf capturable=True, params and state_steps must be on supported devices: .rralphavalue)r<compiler is_compilingr allzipAssertionErrorjit is_scriptingraddrP view_as_realmul_addcmul_sqrt_clonediv_view_as_complexadd_)rrFrGrHrIrrrr rrrrWparamrOrMrNr1stddeltarss @r-_single_tensor_adadeltars" >> & & ( (Z'H( $ v4@   ![\x[yyz{  99 ! ! # # ^47{ D50ZD  #t$ 1 88E86D   E " "++J7J**95I%%d+D%%dC%@nnS!'') c"((* KKME 3T"s$$UES$A   E " "))%0E 5 $15r/c ^U (a [S5e[RR5(dAU (a:[ SS9m[ U4Sj[ XSS955(d[STS35e[U5S :Xag[U5n[R"XX#U/5n U R5GHuunnnnnn[[[U5n[[[U5n[[[U5n[[[U5n[[[U5nU (a[UUUU5 [RR5(d>US R (a*[R""U[R$"S S S 9S S 9 O[R""US5 U (a[R&"U5nUS :wa4U (a[R""UUUS 9 O[R("UUUS 9n[R*"UU5 [R,"UUUSU- S9 [R("UU5n[R."U5 [R("UU5n[R."U5 [R0"UU5 [R*"UU5 [R*"UU5 [R,"UUUSU- S9 U (aQ[3U[R5(a2[R*"UU*5 [R""UU5 GM[R""UUU*S 9 GM g)Nz#_foreach ops don't support autogradFrlc3># UHTupURRURR:H=(a URRT;v MV g7fr`rorqs r-rt)_multi_tensor_adadelta..IrvrwTrxrzr{rr$cpu)r4r|rr~)rr<rrr rrr;rr"_group_tensors_by_device_and_dtypevaluesrrhrris_cpu _foreach_add_r? _foreach_neg _foreach_add _foreach_mul__foreach_addcmul__foreach_sqrt_ _foreach_div_r%)rrFrGrHrIrrrr rrrrWgrouped_tensorsdevice_params_ device_grads_device_square_avgs_device_acc_deltas_device_state_steps__ device_params device_gradsdevice_square_avgsdevice_acc_deltasdevice_state_stepsrdeltasrss @r-_multi_tensor_adadeltar1s BCC >> & & ( (Z'H( $ v4@   ![\x[yyz{  6{a BBBB  =O  " " $   T&\>: DL-8 !$v,0CD f/AB!$v,0CD  |-?AR ~~**,,1CA1F1M1M   "ELLU$C3     2A 6  --l;L 1 ##L-|T$11 -|   .4   l!c'   !3S9 S!##$5s; V$ FC( FL1 -s3  166SQ *R66    ,    v 6    vbS Aq %r/)single_tensor_fnr!c [RR5(d"[SU55(d [ S5eUc [ XSS9upU(a.[R R5(a [ S5eU(a*[R R5(d[nO[nU"UUUUUU U U U U UUUS9 g)znFunctional API that performs Adadelta algorithm computation. See :class:`~torch.optim.Adadelta` for details. c3V# UHn[U[R5v M! g7fr`)r%r<r)rrts r-rtadadelta..s!5-8 1ell##[s')zPAPI has changed, `state_steps` argument must contain a list of singleton tensorsNF) use_fusedz6torch.jit.script not supported with foreach optimizers)rrrr rrrrW) r<rrrrSrrrrr)rrFrGrHrIrr!rrWrrrr rrfuncs r-rrs6 >> & & ( (5-8522 ^  1 e 599))++STTuyy--//%&   !%r/)FNFF)typingrrr<r optimizerrrr r r r r rrrrrrr__all__r__doc__rhr>rerrrrKr/r-rs $ z "ayaJ8       90 j9% L9% <9%f9%V 9% f 9% 9% 9% 9%9%9%9%9%9% 9%xgB LgB <gBfgBV gB f gB gB gB gBgBgBgBgBgB gBT 1HI = L= <=f=V = f ==D[=== = = = !="#=$ %=J=r/