[Scipy-svn] r6891 - trunk/scipy/optimize

scipy-svn@scip... scipy-svn@scip...
Sun Nov 14 04:01:16 CST 2010


Author: rgommers
Date: 2010-11-14 04:01:15 -0600 (Sun, 14 Nov 2010)
New Revision: 6891

Modified:
   trunk/scipy/optimize/__init__.py
   trunk/scipy/optimize/cobyla.py
   trunk/scipy/optimize/lbfgsb.py
   trunk/scipy/optimize/minpack.py
   trunk/scipy/optimize/optimize.py
   trunk/scipy/optimize/slsqp.py
   trunk/scipy/optimize/tnc.py
Log:
DOC: merge wiki edits for optimize module.

Modified: trunk/scipy/optimize/__init__.py
===================================================================
--- trunk/scipy/optimize/__init__.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/__init__.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -1,3 +1,137 @@
+"""
+Optimization Tools
+==================
+
+General-purpose Optimization Routines
+-------------------------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   fmin - Nelder-Mead Simplex algorithm #
+   fmin_powell - Powell's (modified) level set method #
+   fmin_cg - Non-linear (Polak-Ribiere) conjugate gradient algorithm ##
+   fmin_bfgs - Quasi-Newton method (Broydon-Fletcher-Goldfarb-Shanno) ##
+   fmin_ncg - Line-search Newton Conjugate Gradient ###
+   leastsq - Minimize the sum of squares of M equations in N unknowns
+
+Constrained Optimizers (Multivariate)
+-------------------------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   fmin_l_bfgs_b - Zhu, Byrd, and Nocedal's constrained optimizer %
+   fmin_tnc - Truncated Newton code %%
+   fmin_cobyla - Constrained optimization by linear approximation
+   fmin_slsqp - Minimization using sequential least-squares programming
+   nnls - Linear least-squares problem with non-negativity constraint
+
+Global Optimizers
+-----------------
+
+.. autosummary::
+   :toctree: generated/
+
+   anneal - Simulated annealing
+   brute - Brute force searching optimizer
+
+Scalar Function Minimizers
+--------------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   fminbound - Bounded minimization of a scalar function
+   brent - 1-D function minimization using Brent method
+   golden - 1-D function minimization using Golden Section method
+   bracket - Bracket a minimum, given two starting points
+
+General-purpose Root-finding Routines
+-------------------------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   fsolve - Non-linear multi-variable equation solver
+
+Scalar Function Solvers
+-----------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   brentq - quadratic interpolation Brent method
+   brenth - Brent method, modified by Harris with hyperbolic extrapolation
+   ridder - Ridder's method
+   bisect - Bisection method
+   newton - Secant method or Newton's method
+   fixed_point - Single-variable fixed-point solver
+
+General-purpose Non-linear Multidimensional Solvers
+---------------------------------------------------
+
+.. autosummary::
+   :toctree: generated/
+
+   broyden1 - Broyden's first method $
+   broyden2 - Broyden's second method $$
+   broyden3 - Broyden's third method $$$
+   broyden_generalized - Generalized Broyden's method &
+   anderson - Extended Anderson method &&
+   anderson2 - The Anderson method &&&
+
+Utility Functions
+-----------------
+
+.. autosummary::
+   :toctree: generated/
+
+   line_search - Return a step that satisfies the strong Wolfe conditions
+   check_grad - Check the supplied derivative using finite differences
+
+Related Software
+----------------
+
+OpenOpt - A BSD-licensed optimization framework (see `<http://openopt.org>`_) that includes: a number of constrained and
+unconstrained solvers from and beyond the scipy.optimize module; unified
+text and graphical output of convergence information; and automatic
+differentiation.
+
+Notes
+-----
+# Uses only function calls
+
+## Can use function and gradient
+
+### Can use function, gradient, and Hessian
+
+% If you use fmin_l_bfgs_b, please cite Zhu, Byrd, and Nocedal's papers; see the function's docstring for references.
+
+%% Originally written by Stephen Nash, adapted to C by Jean-Sebastien Roy.
+
+$ broyden1 is a quasi-Newton-Raphson method for updating an approximate
+Jacobian and then inverting it.
+
+$$ broyden2 is the same as broyden1, but updates the inverse Jacobian
+directly.
+
+$$$ broyden3 is the same as broyden2, but instead of directly computing
+the inverse Jacobian, it remembers how to construct it using vectors, and
+when computing inv(J)*F, it uses those vectors to compute this product,
+thus avoding the expensive NxN matrix multiplication.
+
+& broyden_generalized is the same as broyden2, but instead of
+approximating the full NxN Jacobian, it constructs it at every iteration
+in a way that avoids the NxN matrix multiplication.  This is not as
+precise as broyden3.
+
+&& anderson is the same as broyden_generalized, but (w_0**2)*I is added
+before taking inversion to improve the stability.
+
+&&& anderson2 is the same as anderson, but formulated differently.
+
+"""
 #
 # optimize - Optimization Tools
 #

Modified: trunk/scipy/optimize/cobyla.py
===================================================================
--- trunk/scipy/optimize/cobyla.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/cobyla.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -18,13 +18,14 @@
 
     Parameters
     ----------
-    func : callable f(x, *args)
-        Function to minimize.
+    func : callable
+        Function to minimize. In the form func(x, \\*args).
     x0 : ndarray
         Initial guess.
     cons : sequence
         Constraint functions; must all be ``>=0`` (a single function
-        if only 1 constraint).
+        if only 1 constraint). Each function takes the parameters `x`
+        as its first argument.
     args : tuple
         Extra arguments to pass to function.
     consargs : tuple
@@ -36,7 +37,7 @@
     rhoend :
         Final accuracy in the optimization (not precisely guaranteed).
     iprint : {0, 1, 2, 3}
-        Controls the frequency of output; 0 implies no output.  Deprecated
+        Controls the frequency of output; 0 implies no output.  Deprecated.
     disp : {0, 1, 2, 3}
         Over-rides the iprint interface.  Preferred.
     maxfun : int
@@ -47,6 +48,30 @@
     x : ndarray
         The argument that minimises `f`.
 
+    Examples
+    --------
+    Minimize the objective function f(x,y) = x*y subject
+    to the constraints x**2 + y**2 < 1 and y > 0::
+
+        >>> def objective(x):
+        ...     return x[0]*x[1]
+        ...
+        >>> def constr1(x):
+        ...     return 1 - (x[0]**2 + x[1]**2)
+        ...
+        >>> def constr2(x):
+        ...     return x[1]
+        ...
+        >>> fmin_cobyla(objective, [0.0, 0.1], [constr1, constr2], rhoend=1e-7)
+
+           Normal return from subroutine COBYLA
+
+           NFVALS =   64   F =-5.000000E-01    MAXCV = 1.998401E-14
+           X =-7.071069E-01   7.071067E-01
+        array([-0.70710685,  0.70710671])
+
+    The exact solution is (-sqrt(2)/2, sqrt(2)/2).
+
     """
     err = "cons must be a sequence of callable functions or a single"\
           " callable function."

Modified: trunk/scipy/optimize/lbfgsb.py
===================================================================
--- trunk/scipy/optimize/lbfgsb.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/lbfgsb.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -79,8 +79,9 @@
         calculating the gradient
     iprint : int
         Controls the frequency of output. ``iprint < 0`` means no output.
-    disp : int
-        If zero, then no output.  If positive number, then this over-rides iprint
+    disp : int, optional
+        If zero, then no output.  If positive number, then this over-rides
+        `iprint`.
     maxfun : int
         Maximum number of function evaluations.
 
@@ -93,36 +94,35 @@
     d : dict
         Information dictionary.
 
-        d['warnflag'] is
-            0 if converged,
-            1 if too many function evaluations,
-            2 if stopped for another reason, given in d['task']
-        d['grad'] is the gradient at the minimum (should be 0 ish)
-        d['funcalls'] is the number of function calls made.
+        * d['warnflag'] is
+          - 0 if converged,
+          - 1 if too many function evaluations,
+          - 2 if stopped for another reason, given in d['task']
 
+        * d['grad'] is the gradient at the minimum (should be 0 ish)
+        * d['funcalls'] is the number of function calls made.
 
-   Notes
-   -----
+    Notes
+    -----
+    License of L-BFGS-B (Fortran code):
 
-   License of L-BFGS-B (Fortran code):
+    The version included here (in fortran code) is 2.1 (released in 1997).
+    It was written by Ciyou Zhu, Richard Byrd, and Jorge Nocedal
+    <nocedal@ece.nwu.edu>. It carries the following condition for use:
 
-   The version included here (in fortran code) is 2.1 (released in
-   1997). It was written by Ciyou Zhu, Richard Byrd, and Jorge Nocedal
-   <nocedal@ece.nwu.edu>. It carries the following condition for use:
+    This software is freely available, but we expect that all publications
+    describing work using this software , or all commercial products using it,
+    quote at least one of the references given below.
 
-   This software is freely available, but we expect that all
-   publications describing work using this software, or all
-   commercial products using it, quote at least one of the references
-   given below.
+    References
+    ----------
+    * R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound
+      Constrained Optimization, (1995), SIAM Journal on Scientific and
+      Statistical Computing , 16, 5, pp. 1190-1208.
+    * C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B,
+      FORTRAN routines for large scale bound constrained optimization (1997),
+      ACM Transactions on Mathematical Software, Vol 23, Num. 4, pp. 550 - 560.
 
-   References
-     * R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound
-       Constrained Optimization, (1995), SIAM Journal on Scientific and
-       Statistical Computing , 16, 5, pp. 1190-1208.
-     * C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B,
-       FORTRAN routines for large scale bound constrained optimization (1997),
-       ACM Transactions on Mathematical Software, Vol 23, Num. 4, pp. 550 - 560.
-
     """
     n = len(x0)
 

Modified: trunk/scipy/optimize/minpack.py
===================================================================
--- trunk/scipy/optimize/minpack.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/minpack.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -168,7 +168,8 @@
 def leastsq(func, x0, args=(), Dfun=None, full_output=0,
             col_deriv=0, ftol=1.49012e-8, xtol=1.49012e-8,
             gtol=0.0, maxfev=0, epsfcn=0.0, factor=100, diag=None):
-    """Minimize the sum of squares of a set of equations.
+    """
+    Minimize the sum of squares of a set of equations.
 
     ::
 
@@ -226,7 +227,7 @@
         residual standard deviation to get the covariance of the
         parameter estimates -- see curve_fit.
     infodict : dict
-        a dictionary of optional outputs with the keys::
+        a dictionary of optional outputs with the key s::
 
             - 'nfev' : the number of function calls
             - 'fvec' : the function evaluated at the output
@@ -242,6 +243,7 @@
                      magnitude. Column j of p is column ipvt(j)
                      of the identity matrix.
             - 'qtf'  : the vector (transpose(q) * fvec).
+
     mesg : str
         A string message giving information about the cause of failure.
     ier : int
@@ -253,6 +255,19 @@
     -----
     "leastsq" is a wrapper around MINPACK's lmdif and lmder algorithms.
 
+    cov_x is a Jacobian approximation to the Hessian of the least squares
+    objective function.
+    This approximation assumes that the objective function is based on the
+    difference between some observed target data (ydata) and a (non-linear)
+    function of the parameters `f(xdata, params)` ::
+
+           func(params) = ydata - f(xdata, params)
+
+    so that the objective function is ::
+
+           min   sum((ydata - f(xdata, params))**2, axis=0)
+         params
+
     """
     x0 = array(x0, ndmin=1)
     n = len(x0)

Modified: trunk/scipy/optimize/optimize.py
===================================================================
--- trunk/scipy/optimize/optimize.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/optimize.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -178,7 +178,8 @@
 
 def fmin(func, x0, args=(), xtol=1e-4, ftol=1e-4, maxiter=None, maxfun=None,
          full_output=0, disp=1, retall=0, callback=None):
-    """Minimize a function using the downhill simplex algorithm.
+    """
+    Minimize a function using the downhill simplex algorithm.
 
     Parameters
     ----------
@@ -208,7 +209,7 @@
     allvecs : list
         Solution at each iteration.
 
-    Other Parameters
+    Other parameters
     ----------------
     xtol : float
         Relative error in xopt acceptable for convergence.
@@ -219,7 +220,7 @@
     maxfun : number
         Maximum number of function evaluations to make.
     full_output : bool
-        Set to True if fval and warnflag outputs are desired.
+        Set to True if fopt and warnflag outputs are desired.
     disp : bool
         Set to True to print convergence messages.
     retall : bool
@@ -227,8 +228,8 @@
 
     Notes
     -----
-    Uses a Nelder-Mead simplex algorithm to find the minimum of
-    a function of one or more variables.
+    Uses a Nelder-Mead simplex algorithm to find the minimum of function of
+    one or more variables.
 
     """
     fcalls, func = wrap_function(func, args)
@@ -1422,7 +1423,8 @@
 def fmin_powell(func, x0, args=(), xtol=1e-4, ftol=1e-4, maxiter=None,
                 maxfun=None, full_output=0, disp=1, retall=0, callback=None,
                 direc=None):
-    """Minimize a function using modified Powell's method.
+    """
+    Minimize a function using modified Powell's method.
 
     Parameters
     ----------
@@ -1431,7 +1433,7 @@
     x0 : ndarray
         Initial guess.
     args : tuple
-        Eextra arguments passed to func.
+        Extra arguments passed to func.
     callback : callable
         An optional user-supplied function, called after each
         iteration.  Called as ``callback(xk)``, where ``xk`` is the

Modified: trunk/scipy/optimize/slsqp.py
===================================================================
--- trunk/scipy/optimize/slsqp.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/slsqp.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -66,14 +66,14 @@
     ----------
     func : callable f(x,*args)
         Objective function.
-    x0 : ndarray of float
+    x0 : 1-D ndarray of float
         Initial guess for the independent variable(s).
     eqcons : list
         A list of functions of length n such that
         eqcons[j](x0,*args) == 0.0 in a successfully optimized
         problem.
     f_eqcons : callable f(x,*args)
-        Returns an array in which each element must equal 0.0 in a
+        Returns a 1-D array in which each element must equal 0.0 in a
         successfully optimized problem.  If f_eqcons is specified,
         eqcons is ignored.
     ieqcons : list
@@ -81,7 +81,7 @@
         ieqcons[j](x0,*args) >= 0.0 in a successfully optimized
         problem.
     f_ieqcons : callable f(x0,*args)
-        Returns an array in which each element must be greater or
+        Returns a 1-D ndarray in which each element must be greater or
         equal to 0.0 in a successfully optimized problem.  If
         f_ieqcons is specified, ieqcons is ignored.
     bounds : list

Modified: trunk/scipy/optimize/tnc.py
===================================================================
--- trunk/scipy/optimize/tnc.py	2010-11-14 10:00:53 UTC (rev 6890)
+++ trunk/scipy/optimize/tnc.py	2010-11-14 10:01:15 UTC (rev 6891)
@@ -83,21 +83,22 @@
              messages=MSG_ALL, maxCGit=-1, maxfun=None, eta=-1,
              stepmx=0, accuracy=0, fmin=0, ftol=-1, xtol=-1, pgtol=-1,
              rescale=-1, disp=None):
-    """Minimize a function with variables subject to bounds, using
+    """
+    Minimize a function with variables subject to bounds, using
     gradient information.
 
     Parameters
     ----------
-    func : callable func(x, *args)
+    func : callable ``func(x, *args)``
         Function to minimize.  Should return f and g, where f is
         the value of the function and g its gradient (a list of
         floats).  If the function returns None, the minimization
         is aborted.
     x0 : list of floats
         Initial estimate of minimum.
-    fprime : callable fprime(x, *args)
+    fprime : callable ``fprime(x, *args)``
         Gradient of func. If None, then func must return the
-        function value and the gradient (f,g = func(x, *args)).
+        function value and the gradient (``f,g = func(x, *args)``).
     args : tuple
         Arguments to pass to function.
     approx_grad : bool
@@ -164,7 +165,7 @@
         The solution.
     nfeval : int
         The number of function evaluations.
-    rc :
+    rc : int
         Return code as defined in the RCSTRINGS dict.
 
     """



More information about the Scipy-svn mailing list