[Numpy-discussion] MapIter api

Charles R Harris charlesr.harris@gmail....
Tue Apr 23 17:29:00 CDT 2013


On Tue, Apr 23, 2013 at 4:06 PM, Sebastian Berg
<sebastian@sipsolutions.net>wrote:

> On Tue, 2013-04-23 at 17:08 -0400, Frédéric Bastien wrote:
> > Hi,
> >
> > this is currently used in Theano! In fact, it is a John S. that
> > implemented it in NumPy to allow fast gradient of the advanced
> > indexing in Theano. It allow code like:
> >
> >
> > matrix1[vector1, vector2] += matrix2
> >
> Yes, I had missed that and thought maybe nobody actually used it yet. I
> gave some points why I think there should be some changes in the
> original pull request [1]. Mostly I think it would make sense (also a
> lot for theano) to rewrite it with the new iterators and expose the
> subspace more directly. That would give vast speedups for mixed
> fancy/non-fancy indices.
>
> But if this is useful to you, I guess one can also just create a new one
> if someone finds time, leaving the old MapIter deprecated and
> unmaintained.
>
> [1] https://github.com/numpy/numpy/pull/377
>
> > where there is duplicate indices in the vector
> >
> > In looking at the code, I saw it use at least those part of the
> > interface.
> >
> > PyArrayMapIterObject
> > PyArray_MapIterNext
> > PyArray_ITER_NEXT
> > PyArray_MapIterSwapAxes
> > PyArray_BroadcastToShape
> >
>
> There is likely no reason for changing these, but improving MapIter
> would likely break binary compatibility because of struct access.
>
> - Sebastian
> >
> > I lost the end of this discussion, but I think this is not possible in
> > NumPy as there was not an agreement to include that. But I remember a
> > few other user on this list asking for this(and they where Theano user
> > to my knowledge).
> >
> >
> > So I would prefer that you don't remove the part that we use for the
> > next 1.8 release.
> >
> > thanks
> >
> > Frédéric
> >
> >
> >
> > On Tue, Apr 16, 2013 at 9:54 AM, Nathaniel Smith <njs@pobox.com>
> > wrote:
> >         On Mon, Apr 15, 2013 at 5:29 PM, Sebastian Berg
> >         <sebastian@sipsolutions.net> wrote:
> >         > Hey,
> >         >
> >         > the MapIter API has only been made public in master right?
> >         So it is no
> >         > problem at all to change at least the mapiter struct, right?
> >         >
> >         > I got annoyed at all those special cases that make things
> >         difficult to
> >         > get an idea where to put i.e. to fix the boolean array-like
> >         stuff. So
> >         > actually started rewriting it (and I already got one big
> >         function that
> >         > does all index preparation -- ok it is untested but its
> >         basically
> >         > there).
> >         >
> >         > I would guess it is not really a big problem even if it was
> >         public for
> >         > longer, since you shouldn't do those direct struct access
> >         probably? But
> >         > just checking.
> >
> >
> >         Why don't we just make the struct opaque, i.e., just declare
> >         it in the
> >         public header file and move the actual definition to an
> >         internal
> >         header file?
> >
> >         If it's too annoying I guess we could even make it non-public,
> >         at
> >         least in 1.8 -- IIRC it's only there so we can use it in
> >         umath, and
> >         IIRC the patch to use it hasn't landed yet. Or we could just
> >         merge
> >         umath and multiarray into a single .so, that would save a
> >         *lot* of
> >         annoying fiddling with the public API that doesn't actually
> >         serve any
> >         purpose.
> >
>

Does this have any overlap with https://github.com/numpy/numpy/pull/2821 ?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20130423/bb459f85/attachment.html 


More information about the NumPy-Discussion mailing list