[Numpy-discussion] Notes on transitioning to numarray.

Todd Miller jmiller at stsci.edu
Mon Nov 10 09:38:04 CST 2003


Nice job Tim!

I'll enter the bugs individually and collectively on source forge.  From
the looks of it,  it'll be a while before they're all sorted out.

Best regards,
Todd

On Mon, 2003-11-10 at 11:26, Tim Hochberg wrote:
> [There is a html version of this not at 
> http://starship.python.net/~hochberg/conversion.html]
> 
> ====================================
> Numeric to numarray Conversion Notes
> ====================================
> 
> I finally bit the bullet over the last few days and moved my current
> project from using Numeric to using  numarray. This was a reasonably
> large undertaking, involving the modification of in excess of thirty
> modules. For the most part it went smoothly, requiring only the
> replacement of ``import Numeric`` with ``import numarray`` [1]_.
> However, in the course of the move I ran into several bugs, as well
> as quite a few things that may be bugs or may be deliberate changes
> from Numeric. I took some notes as the conversion progressed which I
> will attempt to render here in some halfway decipherable form.
> 
> There are still some interoperability issues with Numeric and numarray,
> not all of which I took the time to track down. In many cases, I solved
> problems that cropped up in the conversion simply by converting some more
> of the code and thus reducing the amount of mixed operations. The ones that
> I did track down are reported below.
> 
> I'm using numarray 0.7 with Python 2.3 on Windows XP.
> 
> 
> .. [1] Actually, to be strictly accurate, I replaced ``import Numeric as 
> np`` with ``import numarray as na`` and
>         then replaced ``np`` with ``na``.
> 
> 
> Bugs
> ====
> 
> The following few things are almost certainly bugs. I haven't had
> time to dig into them in any depth, but I have tried to reduce them
> each to a small failing case:
> 
>     1. Copying a slice of an array onto a different slice of the same
>     array fails.
>     
>     >>> y = na.arange(4)
>     >>> y[1:] = y[:-1]
>     >>> y # Should be array([0, 0, 1, 2])
>     array([0, 0, 0, 0])
>     
>     
>     2. ``sqrt``, ``Power``, `and ``**`` all fail on complex zero (0j).
>     
>     >>> y = na.arange(4) + 0j
>     >>> na.sqrt(y)
>     Warning: Encountered invalid numeric result(s)  in sqrt
>     Warning: Encountered divide by zero(s)  in sqrt
>     Warning: Encountered invalid numeric result(s)  in not_equal
>     Warning: Encountered invalid numeric result(s)  in not_equal
>     array([-1.#IND    -1.#INDj,  1.        +0.j    ,  1.41421356+0.j    ,
>             1.73205081+0.j    ])
>     
>     And similarly for ``power`` and ``**``. Note that in addition to
>     the warnings, the value for the sqrt(0j) is incorrect.
>     
>     
>     3. Mixing arrays and lists in the constructor of array can cause
>     it to fail:
>     
>     >>> a = na.array([na.array([])]*3)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\lib\site-packages\numarray\numarraycore.py", 
> line 288, in array
>         return fromlist(sequence, type, shape)
>       File "C:\Python23\lib\site-packages\numarray\numarraycore.py", 
> line 175, in fromlist
>         arr = _gen.concatenate(l)
>       File "C:\Python23\Lib\site-packages\numarray\generic.py", line 
> 1008, in concatenate
>         return _concat(arrs)
>       File "C:\Python23\Lib\site-packages\numarray\generic.py", line 
> 998, in _concat
>         dest[ix:ix+a._shape[0]]._copyFrom(a)
>     libnumarray.error: copy4bytes: access beyond buffer. offset=3 
> buffersize=0
> 
>     ``shape`` fails in the same way. Probably other functions as well.
>     
>     
>     4. ``linear_algebra.determinant`` returns a length-1 vector when it 
> should return a scalar.
>     (Actually, I believe it sometimes returns a scalar and sometimes a 
> length-1 vector, but
>     I can't find a test case to reproduce that).
>     
>     >>> a = na.reshape(na.arange(4), (2,2))
>     >>> la.determinant(a)
>     array([-2.])
>     
>     
>     5. Assigning a Numeric slice to a numarray array fails:
>     
>     >>> a = na.arange(3)
>     >>> b = Numeric.arange(3)
>     >>> a[:] = b
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\Lib\site-packages\numarray\generic.py", line 
> 505, in _slicedIndexing
>         retarr._copyFrom(value)
>     TypeError: argument is not array or number
> 
>     
> Probable Bugs
> =============
> 
> Now on to things that are probably bugs, but it's possible that they
> represent deliberate changes from Numeric's behavior.
> 
>     6. numarray.dot doesn't accept scalars, Numeric.dot does.
>     
>     >>> na.dot(1,1)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\lib\site-packages\numarray\numarraycore.py", 
> line 939, in dot
>         return ufunc.innerproduct(a, _gen.swapaxes(inputarray(b), -1, -2))
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 1892, 
> in innerproduct
>         if a._shape[-1] != b._shape[-1]:
>     IndexError: tuple index out of range
> 
> 
>     7. na.searchsorted does not accept scalars for its second argument. 
> It always
>     takes and returns vectors.
>     
>     >>> na.searchsorted(na.arange(5), 1.5)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 1821, 
> in searchsorted
>         outarr = _nc.NumArray(shape=len(values), type=Long)
>     ValueError: Rank-0 array has no length.
>     >>> na.searchsorted(na.arange(5), [1.5])
> 
> 
>     8. add.reduce takes dim as a keyword argument instead of axis. It is 
> documented_ to take axis.
>     I imagine this applies to other opp.reduce methods as well.
>     
>     .. _documented: 
> http://stsdas.stsci.edu/numarray/Doc/node30.html#SECTION035120000000000000000
> 
> 
>     9. ``where`` and probably other functions do not appear to use 
> asarray on all of their
>     arguments. As a result, nonnumarray sequences are not directly 
> usable in these functions as
>     they are in their Numeric equivalents. In particular, lists, tuples 
> and Numeric arrays do
>     not work:
>     
>     >>> na.where([0,1,1,0], na.arange(4), [-99,-99,-99,-99])
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\Lib\site-packages\numarray\generic.py", line 
> 970, in where
>         return choose(ufunc.not_equal(condition, 0), (y,x), out)
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 1446, 
> in __call__
>         computation_mode, woutarr, cfunc, ufargs = \
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 1462, 
> in _setup
>         convType = _maxPopType(in2)
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 1401, 
> in _maxPopType
>         raise TypeError( "Type of '%s' not supported" % repr(x) )
>     TypeError: Type of '[-99, -99, -99, -99]' not supported
> 
> 
>     10. ``take`` now requires a keyword argument for axis. Attempting 
> the to specify the axis
>     with a nonkeyword arg results in strange behavior. The docs don't 
> appear to describe
>     this behavior:
>     
>     >>> a = na.reshape(na.arange(9), (3,3))
>     >>> a
>     array([[0, 1, 2],
>            [3, 4, 5],
>            [6, 7, 8]])
>     >>> na.take(a, [0,1], 1)
>     array([1, 4])
>     >>> na.take(a, [0,1], axis=1)
>     array([[0, 1],
>            [3, 4],
>            [6, 7]])
> 
> 
>     11. ``argmax`` returns shape () arrays instead of scalars when used 
> on 1D arrays. These cannot be used
>     to index lists
> 
>     >>> a = na.arange(9)
>     >>> i = na.argmax(a)
>     >>> a[i]
>     array(8)
>     >>> range(9)[i]
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>     TypeError: list indices must be integers
>     >>> i
>     array(8)
> 
> 
> 
> 
> Non Bugs
> ========
> 
> These are some things that would probably not be considered bugs, but 
> that I'd like to mention
> because they either tripped me up or made the conversion and thus my 
> life over the last few
> days more difficult than it needed to be. Let the whining begin.
> 
>     
>     12. ``anArray.conjugate()`` acts in place. 
> ``aComplexNumber.conjugate()`` returns a new
>     number. This seems like a very bad state of affairs. 
> ``anArray.conjugate()`` should be
>     renamed.
>     
>     >>> zarray = na.arange(4) * 1j
>     >>> zarray
>     array([ 0.+0.j,  0.+1.j,  0.+2.j,  0.+3.j])
>     >>> zarray.conjugate()
>     >>> zarray
>     array([ 0.+0.j,  0.-1.j,  0.-2.j,  0.-3.j])
>     >>> z = 1+1j
>     >>> z.conjugate()
>     (1-1j)
>     >>> z
>     (1+1j)    
> 
> 
>     13. ``Error.popMode`` should raise an error if the last mode is 
> popped off the stack.
>     Currently the error gets raised the next time a numeric operation is 
> used which may
>     be far away from the inadvertent pop.
> 
>     >>> na.Error.popMode() # Error should be here
>     _NumErrorMode(overflow='warn', underflow='warn', 
> dividebyzero='warn', invalid='warn')
>     >>> zarray /= 0 # Not here.
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in ?
>       File "C:\Python23\lib\site-packages\numarray\numarraycore.py", 
> line 704, in __idiv__
>         ufunc.divide(self, operand, self)
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 120, 
> in handleError
>         modes = Error.getMode()
>       File "C:\Python23\lib\site-packages\numarray\ufunc.py", line 99, 
> in getMode
>         return l[-1]
>     IndexError: list index out of range
> 
> 
>     14. I'm of the opinion that the underflow behavior should default to 
> 'ignore' not 'warn'.
>     Nine times out of ten that's what one wants, and a default that you 
> have to override most
>     of the time is not a good default. It's possible that this opinion 
> may be based on floating
>     point naivet, but it's mine and I'm sticking to it for the time being.
>     
>     15. Now we're getting to very minor things: Argmax's behavior has 
> changed, so that in the
>     case of ties, you will get different results than with Numeric. 
> Perhaps ``>`` became ``>=``?
>     
>     >>> a = na.array([0,1,0,1])
>     >>> na.argmax(a)
>     array(3)
>     >>> np.argmax(a)
>     1
>     
> 
>     16. ``array_repr`` no longer supports the ``suppress_small`` argument.
>     
>     
>     17. ``take`` is really only useful for array types in numarray. In 
> Numeric it was sometimes
>     useful for choosing stuff from lists of objects. My impression is 
> that numarray doesn't try
>     to support objects; that's probably OK since Numeric's support was 
> pretty iffy.
>     
> 
>     18. The fact that array comparison return booleans in numarray broke 
> some of my code because I
>     do some comparisons and then sum the results. In numarray these get 
> summed as Int8 and thus overflow.
>     I don't consider this a problem, I just thought I'd mention it in 
> case someone else runs into it.
> 
> 
> Regards,
> 
> Tim Hochberg
> 
>             
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by: ApacheCon 2003,
> 16-19 November in Las Vegas. Learn firsthand the latest
> developments in Apache, PHP, Perl, XML, Java, MySQL,
> WebDAV, and more! http://www.apachecon.com/
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21030
(410) 338 - 4576





More information about the Numpy-discussion mailing list