EiPjSrSSKJr SSKJr SSKr/SQr"SS\5r S r SS jr SS jr \ r SS jrSS jrSrSrSrSSjrSSjrSSjrSSjrSSjrg)z Functions --------- .. autosummary:: :toctree: generated/ line_search_armijo line_search_wolfe1 line_search_wolfe2 scalar_search_wolfe1 scalar_search_wolfe2 )warn)DCSRCHN)LineSearchWarningline_search_wolfe1line_search_wolfe2scalar_search_wolfe1scalar_search_wolfe2line_search_armijoc\rSrSrSrg)rN)__name__ __module__ __qualname____firstlineno____static_attributes__rW/home/mande/repo/quber/.venv/lib/python3.13/site-packages/scipy/optimize/_linesearch.pyrrsrrcFSUs=:a Us=:aS:d O [S5eg)Nrrz.'c1' and 'c2' do not satisfy'0 < c1 < c2 < 1'.) ValueError)c1c2s r _check_c1_c2rs( ORO!O./ / rc ^^^^^^^^Uc T"T/TQ76nU/mS/mS/mUUUUU4Sjn UUUUUU4Sjn[R"UT5n[XXVUXXU S9 unnnUTSTSUUTS4$)a As `scalar_search_wolfe1` but do a line search to direction `pk` Parameters ---------- f : callable Function `f(x)` fprime : callable Gradient of `f` xk : array_like Current point pk : array_like Search direction gfk : array_like, optional Gradient of `f` at point `xk` old_fval : float, optional Value of `f` at point `xk` old_old_fval : float, optional Value of `f` at point preceding `xk` The rest of the parameters are the same as for `scalar_search_wolfe1`. Returns ------- stp, f_count, g_count, fval, old_fval As in `line_search_wolfe1` gval : array Gradient of `f` at the final point Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. rc<>TS==S- ss'T"TUT--/TQ76$Nrrr)sargsffcpkxks rphiline_search_wolfe1..phiRs( 1 ad"T""rcv>T"TUT--/TQ76TS'TS==S- ss'[R"TST5$rnpdot)rrfprimegcgvalr"r#s rderphi"line_search_wolfe1..derphiVs@ad*T*Q 1 vvd1gr""r)rramaxaminxtol)r(r)r )r r*r#r"gfkold_fval old_old_fvalrrrr/r0r1r$r-derphi0stpfvalr!r+r,s```` ` @@@rrr%sL {R$ 5D B B#### ffS"oG. tT;Cx 1r!udHd1g 55rc [XV5 UcU"S5nUcU"S5nUb#US:wa[SSX#- -U- 5n U S:aSn OSn Sn [XXVXU5n U "XXKS9upp/XU4$)a Scalar function search for alpha that satisfies strong Wolfe conditions alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Function at point `alpha` derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0 old_phi0 : float, optional Value of phi at previous point derphi0 : float, optional Value derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax, amin : float, optional Maximum and minimum step size xtol : float, optional Relative tolerance for an acceptable step. Returns ------- alpha : float Step size, or None if no suitable step was found phi : float Value of `phi` at the new point `alpha` phi0 : float Value of `phi` at `alpha=0` Notes ----- Uses routine DCSRCH from MINPACK. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1`` as described in [1]_. References ---------- .. [1] Nocedal, J., & Wright, S. J. (2006). Numerical optimization. In Springer Series in Operations Research and Financial Engineering. (Springer Series in Operations Research and Financial Engineering). Springer Nature. r?)\(@d)phi0r5maxiter)rminr)r$r-r=old_phi0r5rrr/r0r1alpha1r>dcsrchr6phi1tasks rr r dsj |2w*1 S&$/27:; A:FG CT :F"7Ct d?rc :^^^^^ ^^^^^^S/mS/mS/mS/mUUUUU4Sjn UmUUUUUUU4SjmUc T"T/TQ76n[R"UT5nT b UU UUUU4SjnOSn[U TXVXXXS9 unnnnUc[S[SS 9 OTSnUTSTSUUU4$) a Find alpha that satisfies strong Wolfe conditions. Parameters ---------- f : callable f(x,*args) Objective function. myfprime : callable f'(x,*args) Objective function gradient. xk : ndarray Starting point. pk : ndarray Search direction. The search direction must be a descent direction for the algorithm to converge. gfk : ndarray, optional Gradient value for x=xk (xk being the current parameter estimate). Will be recomputed if omitted. old_fval : float, optional Function value for x=xk. Will be recomputed if omitted. old_old_fval : float, optional Function value for the point preceding x=xk. args : tuple, optional Additional arguments passed to objective function. c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size extra_condition : callable, optional A callable of the form ``extra_condition(alpha, x, f, g)`` returning a boolean. Arguments are the proposed step ``alpha`` and the corresponding ``x``, ``f`` and ``g`` values. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha : float or None Alpha for which ``x_new = x0 + alpha * pk``, or None if the line search algorithm did not converge. fc : int Number of function evaluations made. gc : int Number of gradient evaluations made. new_fval : float or None New function value ``f(x_new)=f(x0+alpha*pk)``, or None if the line search algorithm did not converge. old_fval : float Old function value ``f(x0)``. new_slope : float or None The local slope along the search direction at the new value ````, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. The search direction `pk` must be a descent direction (e.g. ``-myfprime(xk)``) to find a step length that satisfies the strong Wolfe conditions. If the search direction is not a descent direction (e.g. ``myfprime(xk)``), then `alpha`, `new_fval`, and `new_slope` will be None. Examples -------- >>> import numpy as np >>> from scipy.optimize import line_search An objective function and its gradient are defined. >>> def obj_func(x): ... return (x[0])**2+(x[1])**2 >>> def obj_grad(x): ... return [2*x[0], 2*x[1]] We can find alpha that satisfies strong Wolfe conditions. >>> start_point = np.array([1.8, 1.7]) >>> search_gradient = np.array([-1.0, -1.0]) >>> line_search(obj_func, obj_grad, start_point, search_gradient) (1.0, 2, 1, 1.1300000000000001, 6.13, [1.6, 1.4]) rNc<>TS==S- ss'T"TUT--/TQ76$rr)alpharr r!r"r#s rr$line_search_wolfe2..phis( 1 ebj(4((rc>TS==S- ss'T"TUT--/TQ76TS'UTS'[R"TST5$rr')rGrr*r+r, gval_alphar"r#s rr-"line_search_wolfe2..derphi#sI 1 ebj040Q 1 vvd1gr""rcP>TSU:waT"U5 TUT--nT"XUTS5$)Nrr) rGr$xr-extra_conditionr,rJr"r#s rextra_condition2,line_search_wolfe2..extra_condition20s6!}%u URZA"5S$q': :r)r>*The line search algorithm did not converge stacklevel)r(r)r rr)r myfprimer#r"r2r3r4rrrr/rNr>r$r5rO alpha_starphi_star derphi_starr-r!r*r+r,rJs` `` ` ` @@@@@@rrrs| B B 6DJ))F##  {R$ffS"oG" ; ;  2F b 3//J(K 9 1 .1g r!ubeXx DDrc [XV5 UcU"S5nUcU"S5nSn UbUS:wa[SSX#- -U- 5n OSn U S:aSn Ub [X5n U"U 5n Un UnUcSn[U 5HnU S:XdUb/X:a*SnUnUnSnU S:XaSnOSS U3-n[U[S S 9 OUS:nXX[-U--:d X:aU(a[ XU XXX$XVU5 unnn OU"U 5n[ U5U*U-::aU"X5(aU nU nUn O[US:a[ XU U UXX$XVU5 unnn O=S U -nUb [UU5nU n Un U n U"U 5n UnM U nU nSn[S [S S 9 UUUU4$) aFind alpha that satisfies strong Wolfe conditions. alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Objective scalar function. derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0. old_phi0 : float, optional Value of phi at previous point. derphi0 : float, optional Value of derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size. extra_condition : callable, optional A callable of the form ``extra_condition(alpha, phi_value)`` returning a boolean. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha_star : float or None Best alpha, or None if the line search algorithm did not converge. phi_star : float phi at alpha_star. phi0 : float phi at 0. derphi_star : float or None derphi at alpha_star, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. Nr9rr:r;cg)NTr)rGr$s rrN-scalar_search_wolfe2..extra_conditionsrz7Rounding errors prevent the line search from convergingz4The line search algorithm could not find a solution zless than or equal to amax: rRrSrQ)rr?rangerr_zoomabs)r$r-r=r@r5rrr/rNr>alpha0rAphi_a1phi_a0 derphi_a0irVrWrXmsgnot_first_iteration derphi_a1alpha2s rr r Isp |2w* F1 S&$/27:; z V" [FFI 7^ Q;4+ JHDK{OL4TF;< 'A 6 !e BK'11 1  #6ff$"R_F .J+ 6N  Nrc'k )v..# !'  Nff$i"R_F .J+ V  &FV cj   9 1 . x{ 22rc r[R"SSSS9 UnX0- nXP- n X-S-X- -n [R"S5n U S-U S'US-*U S'U S-*U S'US-U S '[R"U [R"XA- Xx-- Xa- Xy-- /5R 55upX-n X-n X-SU -U-- nX *[R "U5-SU -- -nS S S 5 [R"W5(dg U$![a S S S 5 g f=f!,(df  ND=f) z Finds the minimizer for a cubic polynomial that goes through the points (a,fa), (b,fb), and (c,fc) with derivative at a of fpa. If no minimizer can be found, return None. raisedivideoverinvalidrR)rRrR)rr)rr)rr)rrN) r(errstateemptyr)asarrayflattensqrtArithmeticErrorisfinite)afafpabfbcr!Cdbdcdenomd1ABradicalxmins r _cubicminrsN G'7 C ABBWNbg.E&!BQwBtHaxBtHaxBtHQwBtHVVB BGaf,<,.Gaf,<,>!??FwyJFQ JA JAea!eai'GRWWW--!a%88D! D& ;;t   K  % D C" # D Cs)D(CD D%D($D%%D(( D6c[R"SSSS9 UnUnX0S-- nXE- Xg-- Xw-- nXSU-- - n SSS5 [R"W 5(dgU $![a SSS5 gf=f!,(df  ND=f)zz Finds the minimizer for a quadratic polynomial that goes through the points (a,fa), (b,fb) with derivative at a of fpa. rirjr:@N)r(rortru) rvrwrxryrzDr|r}rrs r_quadminrs G'7 C AAWB!&RW-AC!G}$D D ;;t   K   D C  D Cs(A;#A## A8-A;7A88A;; B c Sn Sn SnSnUnSnX- nUS:aXnnOXnnU S:aUU-n[XXAUUU5nU S:XdWbUUW- :d UUU-:a/UU-n[XXAU5nUbUUU- :d UUU-:aUSU--nU"U5nUXyU-U--:dUU:a UnUnUnUnONU"U5n[U5U *U-::aU "UU5(aUnUnUnO1UX- -S:a UnUnUnUnOUnUnUnUnUnU S- n X:aSnSnSnOMUUU4$)zZoom stage of approximate linesearch satisfying strong Wolfe conditions. Part of the optimization algorithm in `scalar_search_wolfe2`. Notes ----- Implements Algorithm 3.6 (zoom) in Wright and Nocedal, 'Numerical Optimization', 1999, pp. 61. rg?皙?N?r)rrr^)a_loa_hiphi_lophi_hi derphi_lor$r-r=r5rrrNr>rcdelta1delta2phi_reca_recdalpharvrycchka_jqchkphi_aj derphi_aja_starval_star valprime_stars rr]r]sG A F FG E  A:qAqqA EF?DD)6!7,C F q4xS1t8^F?D4&AC qv34<SZ'S TsF7N* *&0@GEDFs I9~"W,f1M1M! ) $+&!+  DF!I Q KFH M A B 8] **rc^^^^^ [R"T5mS/m UUU UU4SjnUc U"S5n OUn [R"UT5n [XXUS9upU T SU 4$)aMinimize over alpha, the function ``f(xk+alpha pk)``. Parameters ---------- f : callable Function to be minimized. xk : array_like Current point. pk : array_like Search direction. gfk : array_like Gradient of `f` at point `xk`. old_fval : float Value of `f` at point `xk`. args : tuple, optional Optional arguments. c1 : float, optional Value to control stopping criterion. alpha0 : scalar, optional Value of `alpha` at start of the optimization. Returns ------- alpha f_count f_val_at_alpha Notes ----- Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 rc<>TS==S- ss'T"TUT--/TQ76$rr)rArr r!r"r#s rr$line_search_armijo..phis( 1 fRi'$''rr9)rr_)r( atleast_1dr)scalar_search_armijo)r r#r"r2r3rrr_r$r=r5rGrCr!s``` ` @rr r osnD r B B((2wffS"oG&s'.46KE "Q% rc :[XX#XEUUS9nUSUSSUS4$)z0 Compatibility wrapper for `line_search_armijo` )rrr_rrrR)r ) r r#r"r2r3rrr_rs rline_search_BFGSrs4 1"82"( *A Q41q!A$ rc<U"U5nXaX4-U--::aXF4$U*US--S- Xa- X$-- - nU"U5nXX7-U--::aXx4$Xu:aUS-US--Xt- -n US-X- X'-- -US-Xa- X$-- -- n X- n US-*X- X'-- -US-Xa- X$-- --n X- n U *[R"[U S-SU -U-- 55-SU -- n U"U 5n XX<-U--::aX4$X|- US- :d SX- - S:aUS- n UnU nUnU nXu:aMSU4$)aMinimize over alpha, the function ``phi(alpha)``. Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 alpha > 0 is assumed to be a descent direction. Returns ------- alpha phi1 rRrrng@rgQ?N)r(rsr^)r$r=r5rr_r0rarAr`factorrvryrgphi_a2s rrrs[F  ')))~Z&!) #c )V]W=M-M NF [F7***~ -VQY&&-8 AI7 8 AI7 8 9 J QYJ&-'.8 9 AI7 8 9 J"rwws1a4!a%'/#9:;;AFV RYw.. .> ! Ov| +FM0AT/Ic\F+ -0 <rcUSn[U5n Sn Sn Sn XU--n U"U 5upXU-XZS--U-- ::aU n OU S-U-USU -S- U--- nXU-- n U"U 5upXU-X[S--U-- ::aU *n OQU S-U-USU -S- U--- n[R"UXj-Xz-5n [R"UXk-X{-5n MXX4$)a Nonmonotone backtracking line search as described in [1]_ Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. prev_fs : float List of previous merit function values. Should have ``len(prev_fs) <= M`` where ``M`` is the nonmonotonicity window parameter. eta : float Allowed merit function increase, see [1]_ gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position References ---------- [1] "Spectral residual method without gradient information for solving large-scale nonlinear systems of equations." W. La Cruz, J.M. Martinez, M. Raydan. Math. Comp. **75**, 1429 (2006). rrR)maxr(clip)r x_kdprev_fsetagammatau_mintau_maxf_kf_baralpha_palpha_mrGxpfpFpalpha_tpalpha_tms r_nonmonotone_line_search_cruzrs+P "+C LEGG E  Q; 2 uz1C77 7E A:#rQwY]C,?'?@ Q; 2 uz1C77 7HE A:#rQwY]C,?'?@''(G$5w7HI''(G$5w7HI) , b rc Sn Sn Sn XU--nU"U5unnXU-X{S--U-- ::aU n OU S-U-USU -S- U--- nXU-- nU"U5unnXU-X|S--U-- ::aU *n OQU S-U-USU -S- U--- n[R"UX-X-5n [R"UX-X-5n MX-S-nX-XF--U-U- nUnXUUXE4$)a Nonmonotone line search from [1] Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. f_k : float Initial merit function value. C, Q : float Control parameters. On the first iteration, give values Q=1.0, C=f_k eta : float Allowed merit function increase, see [1]_ nu, gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position C : float New value for the control parameter C Q : float New value for the control parameter Q References ---------- .. [1] W. Cheng & D.-H. Li, ''A derivative-free nonmonotone line search and its application to the spectral residual method'', IMA J. Numer. Anal. 29, 814 (2009). rrR)r(r)r rrrr|QrrrrnurrrGrrrrrQ_nexts r_nonmonotone_line_search_chengr2sI^GG E  Q; 2B S5A:-33 3E A:#rQwY]C,?'?@ Q; 2B S5A:-33 3HE A:#rQwY]C,?'?@''(G$5w7HI''(G$5w7HI) .VaZF 17 b F*AA b"a ""r) NNNr-C6??2:0yE>+=)NNNrrrrr) NNNrrrNNr)NNNrrNNr)rrr)rrr)rrr)rrrg333333?)__doc__warningsr_dcsrchrnumpyr(__all__RuntimeWarningrrrr line_searchrr rrr]r rrrrrrrrs  !  //337?C!<6~IM%(27JZ! @DIM57LE^,004/379Q3hD*T+v1h7~DGEREH&*N#r