[Numpy-discussion] Raveling, reshape order keyword unnecessarily confuses index and memory ordering

josef.pktd@gmai... josef.pktd@gmai...
Fri Apr 5 18:27:49 CDT 2013


On Fri, Apr 5, 2013 at 6:09 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
> Hi,
>
> On Fri, Apr 5, 2013 at 12:53 PM, Ralf Gommers <ralf.gommers@gmail.com> wrote:
>>
>>
>>
>> On Fri, Apr 5, 2013 at 9:21 PM, Matthew Brett <matthew.brett@gmail.com>
>> wrote:
>>>
>>> Hi,
>>>
>>> On Fri, Apr 5, 2013 at 3:09 PM, Ralf Gommers <ralf.gommers@gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > On Fri, Apr 5, 2013 at 5:13 PM, Matthew Brett <matthew.brett@gmail.com>
>>> > wrote:
>>> >>
>>> >> Hi,
>>> >>
>>> >> On Fri, Apr 5, 2013 at 2:20 AM, Sebastian Berg
>>> >> <sebastian@sipsolutions.net> wrote:
>>> >> > Hey
>>> >> >
>>> >> > On Thu, 2013-04-04 at 14:20 -0700, Matthew Brett wrote:
>>> >> >> Hi,
>>> >> >>
>>> >> >> On Tue, Apr 2, 2013 at 4:32 AM, Nathaniel Smith <njs@pobox.com>
>>> >> >> wrote:
>>> >> >> <snip>
>>> >> >> > Maybe we should go through and rename "order" to something more
>>> >> >> > descriptive
>>> >> >> > in each case, so we'd have
>>> >> >> >   a.reshape(..., index_order="C")
>>> >> >> >   a.copy(memory_order="F")
>>> >> >> > etc.?
>>> >> >>
>>> >> >> I'd like to propose this instead:
>>> >> >>
>>> >> >> a.reshape(..., order="C")
>>> >> >> a.copy(layout="F")
>>> >> >>
>>> >> >
>>> >> > I actually like this, makes the point clearer that it has to do with
>>> >> > memory layout and implies contiguity, plus it is short and from the
>>> >> > numpy perspective copy, etc. are the ones that add additional info to
>>> >> > "order" and not reshape (because IMO memory order is something new
>>> >> > users
>>> >> > should not worry about at first). A and K orders will still have
>>> >> > their
>>> >> > quirks with np.array and copy=True/False, but for many functions they
>>> >> > are esoteric anyway.
>>> >> >
>>> >> > It will be one hell of a deprecation though, but I am +0.5 for adding
>>> >> > an
>>> >> > alias for now (maybe someone knows an even better name?), but I think
>>> >> > that in this case, it probably really is better to wait with actual
>>> >> > deprecation warnings for a few versions, since it touches a *lot* of
>>> >> > code. Plus I think at the point of starting deprecation warnings (and
>>> >> > best earlier) numpy should provide an automatic fixer script...
>>> >> >
>>> >> > The only counter point that remains for me is the difficulty of
>>> >> > deprecation, since I think the new name idea is very clean. And this
>>> >> > is
>>> >> > unfortunately even more invasive then the index_order proposal.
>>> >>
>>> >> I completely agree that we'd have to be gentle with the change.  The
>>> >> problem we'd want to avoid is people innocently using 'layout' and
>>> >> finding to their annoyance that the code doesn't work with other
>>> >> people's numpy.
>>> >>
>>> >> How about:
>>> >>
>>> >> Step 1:  'order' remains as named keyword, layout added as alias,
>>> >> comment on the lines of "layout will become the default keyword for
>>> >> this option in later versions of numpy; please consider updating any
>>> >> code that does not need to remain backwards compatible'.
>>> >>
>>> >> Step 2: default keyword becomes 'layout' with 'order' as alias,
>>> >> comment like "order is an alias for 'layout' to maintain backwards
>>> >> compatibility with numpy <= 1.7.1', please update any code that does
>>> >> not need to maintain backwards compatibility with these numpy
>>> >> versions'
>>> >>
>>> >> Step 3: Add deprecation warning for 'order', "order will be removed as
>>> >> an alias in future versions of numpy"
>>> >>
>>> >> Step 4: (distant future) Remove alias
>>> >>
>>> >> ?
>>> >
>>> >
>>> > A very strong -1 from me. Now we're talking about deprecation warnings
>>> > and a
>>> > backwards compatibility break after all. I thought we agreed that this
>>> > was a
>>> > very bad idea, so why are you proposing it now?
>>> >
>>> > Here's how I see it: deprecation of "order" is a no go. Therefore we
>>> > have
>>> > two choices here:
>>> > 1. Simply document the current "order" keyword better and leave it at
>>> > that.
>>> > 2. Add a "layout" (or "index_order") keyword, and live with both "order"
>>> > and
>>> > "layout" keywords forever.
>>> >
>>> > (2) is at least as confusing as (1), more work and poor design.
>>> > Therefore I
>>> > propose to go with (1).
>>>
>>> You are saying that deprecation of 'order' at any stage in the next 10
>>> years of numpy's lifetime is a no go?
>>
>>
>> For something like this? Yes.
>
> You are saying I think that I am wrong in thinking this is an
> important change that will make numpy easier to explain and use in the
> long term.
>
> You'd probably expect me to disagree, and I do.  I think I am right in
> thinking the change is important - I've tried to make that case in
> this thread, as well as I can.
>
>>> I think that is short-sighted and I think it will damage numpy.
>>
>>
>> It will damage numpy to be conservative and not change a name for a little
>> bit of clarity for some people that avoids reading the docs maybe a little
>> more carefully? There's a lot of things that can damage numpy, but this
>> isn't even close in my book. Too few developers, continuous backwards
>> compatibility issues, faster alternative libraries surpassing numpy - that's
>> the kind of thing that causes damage.
>
> We're talked about consensus on this list.  Of course it can be very
> hard to achieve.

So far the consensus is that the documentation needs improvement.

After that ???

my summary:

I still think it's has way too much impact for just a name change.

I think views are unfamiliar coming from another matrix language,
that has a only memory (and copies arrays all the time), and
no single word can help much in "getting it".

Once we understand the distinction between views and memory,
then the dual use of ``order`` has a nice symmetry to it (for me).

95% of all numpy users/usage (a wild guess) work(s) without
any consideration of actual memory.


If there is no consensus for change, then the status-quo prevails
(unless the executive council over-rides).

(I had prepared another message, that got censored, which starts
with
What is the numpy equivalent of Matlab's   x(:)?
quick answer, and long answer ?
)

Josef


>
>>> Believe me, I have as much investment in backward compatibility as you
>>> do.  All the three libraries that I spend a long time maintaining need
>>> to test against old numpy versions - but - for heaven's sake - only
>>> back to numpy 1.2 or numpy 1.3.  We don't support Python 2.5 any more,
>>> and I don't think we need to maintain compatibility with Numeric
>>> either.
>>
>>
>> Really? This is from 3 months ago:
>> http://article.gmane.org/gmane.comp.python.numeric.general/52632. It's now
>> 2013, we are probably dropping numarray compat in 1.8. Not exactly 10 years,
>> but of the same order.
>
> I am happy to make this change over the same time course if you think
> that is necessary.
>
> Cheers,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion


More information about the NumPy-Discussion mailing list