[SciPy-dev] Abstract vectors in optimization

Robert Kern robert.kern@gmail....
Tue Jan 6 21:56:51 CST 2009

2009/1/6 Ben FrantzDale <benfrantzdale@gmail.com>:
> David, Robert, et al.
> I think you understand what I am talking about. This is just about
> optimization (although in general I would argue that functions should take
> as general an interface as possible for the same reasons).

I'm not sure I would push it that far. Premature generalization can be
just as bad as premature optimization. I find that a use case-based
approach is a better principle. Write the code for the problem in
front of you. Generalize when you have a use case for doing so.

Now, we do have a handful of use cases here, but I think we should
have some care before modifying the functions in place. Generalization
could cost us performance in the typical case.

> Attached is all the code I changed to make fmin_cg work with a
> SimpleHilbertVector class.

Hmm, can you show us a .diff of the changes that you made? I'd like to
see more clearly exactly what you changed.

> If people are receptive to this approach, I have a few next questions:
> 1. What is the right interface to provide? Is the interface I described the
> right (Pythonic) way to do it, or is there a better interface? (If it were
> Templated C++, I'd do it with overloaded free functions, but I think it
> needs to/should be done with class methods.)

I still don't think this interface supports the manifold-optimization
use case. The inner product and norm implementations need more
information than just the vectors.

> 2. Which functions should be changed? (fmin_cg, clearly; what else? The full
> list in optimize.py is fmin, fmin_powell, fmin_bfgs, fmin_ncg, fmin_cg, and
> fminbound.)
> 3. How would one deal with numpy arrays? Is it best to wrap them in the
> interface (as shown in the attached code) or is it better to add methods to
> them?

We're not going to add methods to them.

> If the right way is to wrap them, should all the optimization
> functions have that as a first step -- check for numpy arrays and wrap as
> needed?

Possibly, but that could have a significant performance hit that will
need to be tested.

> Given an appropriate interface, the changes to optimize.py are essentially
> search-and-replace and I would be happy to do it.
> Does Travis Oliphant (whose name is on optimize.py) read this list?

Yeah. He's pretty busy this week, though.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco

More information about the Scipy-dev mailing list