[Numpy-discussion] Obscure code in concatenate code path?

Travis Oliphant travis@continuum...
Thu Sep 13 11:51:52 CDT 2012


On Sep 13, 2012, at 11:39 AM, Warren Weckesser wrote:

> 
> 
> On Thu, Sep 13, 2012 at 9:01 AM, Travis Oliphant <travis@continuum.io> wrote:
> 
> On Sep 13, 2012, at 8:40 AM, Nathaniel Smith wrote:
> 
> > On Thu, Sep 13, 2012 at 11:12 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
> >> Hi,
> >>
> >> While writing some tests for np.concatenate, I ran foul of this code:
> >>
> >>    if (axis >= NPY_MAXDIMS) {
> >>        ret = PyArray_ConcatenateFlattenedArrays(narrays, arrays, NPY_CORDER);
> >>    }
> >>    else {
> >>        ret = PyArray_ConcatenateArrays(narrays, arrays, axis);
> >>    }
> >>
> >> in multiarraymodule.c
> >
> > How deeply weird
> 
> 
> This is expected behavior.
> 
> 
> Heh, I guess "expected" is subjective:

"Expected" only in the sense that the current C-API has been intentional for 6 years.   The side-effect of the Python-side being confusing can be changed --- it just hasn't been yet --- the documented approach is to use None.   

A patch to PyArray_AxisConverter might be the answer. 

-Travis




> 
> In [23]: np.__version__
> Out[23]: '1.6.1'
> 
> In [24]: a = zeros((2,2))
> 
> In [25]: b = ones((2,3))
> 
> In [26]: concatenate((a, b), axis=0)  # Expected error.
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> /Users/warren/gitwork/class-material/demo/pytables/<ipython-input-26-7cefb735e507> in <module>()
> ----> 1 concatenate((a, b), axis=0)  # Expected error.
> 
> ValueError: array dimensions must agree except for d_0
> 
> In [27]: concatenate((a, b), axis=1)   # Normal behavior.
> Out[27]: 
> array([[ 0.,  0.,  1.,  1.,  1.],
>        [ 0.,  0.,  1.,  1.,  1.]])
> 
> In [28]: concatenate((a, b), axis=2)   # Cryptic error message.
> ---------------------------------------------------------------------------
> ValueError                                Traceback (most recent call last)
> /Users/warren/gitwork/class-material/demo/pytables/<ipython-input-28-0bce84c34ef1> in <module>()
> ----> 1 concatenate((a, b), axis=2)   # Cryptic error message.
> 
> ValueError: bad axis1 argument to swapaxes
> 
> In [29]: concatenate((a, b), axis=32)   # What the... ?
> Out[29]: array([ 0.,  0.,  0.,  0.,  1.,  1.,  1.,  1.,  1.,  1.])
> 
> 
> I would expect an error, consistent with the behavior when 1 < axis < 32.
> 
> 
> Warren
> 
> 
>  
> It's how the concatenate Python function manages to handle axis=None to flatten the arrays before concatenation.    This has been in NumPy since 1.0 and should not be changed without deprecation warnings which I am -0 on.
> 
> Now, it is true that the C-API could have been written differently (I think this is what Mark was trying to encourage) so that there are two C-API functions and they are dispatched separately from the array_concatenate method depending on whether or not a None is passed in.   But, the behavior is documented and has been for a long time.
> 
> Reference PyArray_AxisConverter (which turns a "None" Python argument into an axis=MAX_DIMS).   This is consistent behavior throughout the C-API.
> 
> -Travis
> 
> 
> 
> 
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
> 
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20120913/b36d07a3/attachment.html 


More information about the NumPy-Discussion mailing list