[Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering

Matthew Brett matthew.brett@gmail....
Fri Apr 5 17:09:14 CDT 2013


Hi,

On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
>
>
>
> On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett <matthew.brett@gmail.com>
> wrote:
>>
>> Hi,
>>
>> On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers <ralf.gommers@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett <matthew.brett@gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg
>> >> <sebastian@sipsolutions.net> wrote:
>> >> > Hey
>> >> >
>> >> > On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote:
>> >> >> Hi,
>> >> >>
>> >> >> On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith <njs@pobox.com>
>> >> >> wrote:
>> >> >> <snip>
>> >> >> > Maybe we should go through and rename "order" to something more
>> >> >> > descriptive
>> >> >> > in each case, so we'd have
>> >> >> >   a.reshape(..., index_order="C")
>> >> >> >   a.copy(memory_order="F")
>> >> >> > etc.?
>> >> >>
>> >> >> I'd like to propose this instead:
>> >> >>
>> >> >> a.reshape(..., order="C")
>> >> >> a.copy(layout="F")
>> >> >>
>> >> >
>> >> > I actually like this, makes the point clearer that it has to do with
>> >> > memory layout and implies contiguity, plus it is short and from the
>> >> > numpy perspective copy, etc. are the ones that add additional info to
>> >> > "order" and not reshape (because IMO memory order is something new
>> >> > users
>> >> > should not worry about at first). A and K orders will still have
>> >> > their
>> >> > quirks with np.array and copy=True/False, but for many functions they
>> >> > are esoteric anyway.
>> >> >
>> >> > It will be one hell of a deprecation though, but I am +0.5 for adding
>> >> > an
>> >> > alias for now (maybe someone knows an even better name?), but I think
>> >> > that in this case, it probably really is better to wait with actual
>> >> > deprecation warnings for a few versions, since it touches a *lot* of
>> >> > code. Plus I think at the point of starting deprecation warnings (and
>> >> > best earlier) numpy should provide an automatic fixer script...
>> >> >
>> >> > The only counter point that remains for me is the difficulty of
>> >> > deprecation, since I think the new name idea is very clean. And this
>> >> > is
>> >> > unfortunately even more invasive then the index_order proposal.
>> >>
>> >> I completely agree that we'd have to be gentle with the change.  The
>> >> problem we'd want to avoid is people innocently using 'layout' and
>> >> finding to their annoyance that the code doesn't work with other
>> >> people's numpy.
>> >>
>> >> How about:
>> >>
>> >> Step 1:  'order' remains as named keyword, layout added as alias,
>> >> comment on the lines of "layout will become the default keyword for
>> >> this option in later versions of numpy; please consider updating any
>> >> code that does not need to remain backwards compatible'.
>> >>
>> >> Step 2: default keyword becomes 'layout' with 'order' as alias,
>> >> comment like "order is an alias for 'layout' to maintain backwards
>> >> compatibility with numpy <= 1.7.1', please update any code that does
>> >> not need to maintain backwards compatibility with these numpy
>> >> versions'
>> >>
>> >> Step 3: Add deprecation warning for 'order', "order will be removed as
>> >> an alias in future versions of numpy"
>> >>
>> >> Step 4: (distant future) Remove alias
>> >>
>> >> ?
>> >
>> >
>> > A very strong -1 from me. Now we're talking about deprecation warnings
>> > and a
>> > backwards compatibility break after all. I thought we agreed that this
>> > was a
>> > very bad idea, so why are you proposing it now?
>> >
>> > Here's how I see it: deprecation of "order" is a no go. Therefore we
>> > have
>> > two choices here:
>> > 1. Simply document the current "order" keyword better and leave it at
>> > that.
>> > 2. Add a "layout" (or "index_order") keyword, and live with both "order"
>> > and
>> > "layout" keywords forever.
>> >
>> > (2) is at least as confusing as (1), more work and poor design.
>> > Therefore I
>> > propose to go with (1).
>>
>> You are saying that deprecation of 'order' at any stage in the next 10
>> years of numpy's lifetime is a no go?
>
>
> For something like this? Yes.

You are saying I think that I am wrong in thinking this is an
important change that will make numpy easier to explain and use in the
long term.

You'd probably expect me to disagree, and I do.  I think I am right in
thinking the change is important - I've tried to make that case in
this thread, as well as I can.

>> I think that is short-sighted and I think it will damage numpy.
>
>
> It will damage numpy to be conservative and not change a name for a little
> bit of clarity for some people that avoids reading the docs maybe a little
> more carefully? There's a lot of things that can damage numpy, but this
> isn't even close in my book. Too few developers, continuous backwards
> compatibility issues, faster alternative libraries surpassing numpy - that's
> the kind of thing that causes damage.

We're talked about consensus on this list.  Of course it can be very
hard to achieve.

>> Believe me, I have as much investment in backward compatibility as you
>> do.  All the three libraries that I spend a long time maintaining need
>> to test against old numpy versions - but - for heaven's sake - only
>> back to numpy 1.2 or numpy 1.3.  We don't support Python 2.5 any more,
>> and I don't think we need to maintain compatibility with Numeric
>> either.
>
>
> Really? This is from 3 months ago:
> http://article.gmane.org/gmane.comp.python.numeric.general/52632. It's now
> 2013, we are probably dropping numarray compat in 1.8. Not exactly 10 years,
> but of the same order.

I am happy to make this change over the same time course if you think
that is necessary.

Cheers,

Matthew


More information about the NumPy-Discussion mailing list