[Numpy-discussion] Revisiting numpy/scipy on 64 bit OSX

Chris Kees christopher.e.kees@usace.army....
Fri Aug 22 09:00:19 CDT 2008


I've been experimenting with both a non-framework, non-universal 64-bit
build and a 4-way universal build of the python (2.6) trunk with numpy
1.1.1. The non-framework 64 build appears to give me exactly the same
results from numpy.test() as the standard 32-bit version (as well as
allowing large arrays like numpy.zeros((1000,1000,1000),'d') ), which is

<unittest._TextTestResult run=1300 errors=0 failures=0>

Our numerical models also seem to run fine with it using 8-16G. The 4-way
universal python gives the same results in 32-bit  but when running in
64-bit I get an error in the tests below, which I haven't had time to look
at.  It also gives the error

>>> a = numpy.zeros((1000,1000,1000),'d')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: dimensions too large.

Anyway, this all goes to say I'm interested in making sure numpy is
dependable in 64-bit on OSX 10.5 and preferably for 4-way universal builds,
but I haven't seen test failures like yours, at least not recently.  In my
experience so far on the Mac it's still important to make sure Xcode and the
OS are up to date (or to use a completely segregated set of consistently
build gnu compilers like  you did) and to make sure the fink and mac  ports
stuff  is not in your path before building python and numpy. Otherwise
configure  and distutils end up making bad choices that I don't have the
skill to root out. I've had various versions of non-framework 64-bit builds
of python running with numpy and Numeric for a while now (a year maybe). The
framework and universal builds of python have only worked recently, and I'm
still not confident in them.

I don't know what's involved in the buildbots. If it's simple, doesn't take
much time to set up, and works behind a tight  firewall I might be able to
set some up. 

Chris


Numpy is installed in
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-package
s/numpy
Numpy version 1.1.1
Python version 2.6b2+ (trunk:65908, Aug 20 2008, 08:47:16) [GCC 4.0.1 (Apple
Inc. build 5484)]
  Found 18/18 tests for numpy.core.tests.test_defmatrix
  Found 3/3 tests for numpy.core.tests.test_errstate
  Found 3/3 tests for numpy.core.tests.test_memmap
  Found 290/290 tests for numpy.core.tests.test_multiarray
  Found 72/72 tests for numpy.core.tests.test_numeric
  Found 36/36 tests for numpy.core.tests.test_numerictypes
  Found 12/12 tests for numpy.core.tests.test_records
  Found 145/145 tests for numpy.core.tests.test_regression
  Found 7/7 tests for numpy.core.tests.test_scalarmath
  Found 2/2 tests for numpy.core.tests.test_ufunc
  Found 16/16 tests for numpy.core.tests.test_umath
  Found 63/63 tests for numpy.core.tests.test_unicode
  Found 4/4 tests for numpy.distutils.tests.test_fcompiler_gnu
  Found 5/5 tests for numpy.distutils.tests.test_misc_util
  Found 2/2 tests for numpy.fft.tests.test_fftpack
  Found 3/3 tests for numpy.fft.tests.test_helper
  Found 25/25 tests for numpy.lib.tests.test__datasource
  Found 10/10 tests for numpy.lib.tests.test_arraysetops
  Found 1/1 tests for numpy.lib.tests.test_financial
  Found 55/55 tests for numpy.lib.tests.test_function_base
  Found 5/5 tests for numpy.lib.tests.test_getlimits
  Found 6/6 tests for numpy.lib.tests.test_index_tricks
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-package
s/numpy/lib/tests/test_io.py:11: SyntaxWarning: assertion is always true,
perhaps remove parentheses?
  assert(c.readlines(),
  Found 15/15 tests for numpy.lib.tests.test_io
  Found 1/1 tests for numpy.lib.tests.test_machar
  Found 4/4 tests for numpy.lib.tests.test_polynomial
  Found 1/1 tests for numpy.lib.tests.test_regression
  Found 49/49 tests for numpy.lib.tests.test_shape_base
  Found 15/15 tests for numpy.lib.tests.test_twodim_base
  Found 43/43 tests for numpy.lib.tests.test_type_check
  Found 1/1 tests for numpy.lib.tests.test_ufunclike
  Found 89/89 tests for numpy.linalg.tests.test_linalg
  Found 3/3 tests for numpy.linalg.tests.test_regression
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/__init__.py", line 126, in test
    return NumpyTest().test(*args, **kw)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 579, in test
    level, verbosity)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 524, in _test_suite_from_all_tests
    verbosity=verbosity)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 417, in _get_suite_list
    suite = obj(mthname)
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/ma/tests/test_core.py", line 36, in __init__
    self.setUp()
  File 
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/ma/tests/test_core.py", line 49, in setUp
    xf = numpy.where(m1, 1.e+20, x)
ValueError: invalid entry in choice array



On 8/21/08 4:15 PM, "Michael Abshoff" <michael.abshoff@googlemail.com>
wrote:

> Hi,
> 
> I have been playing with 64 bit numpy/scipy on OSX 10.5 Intel again. I
> thought everything worked after the last time we discussed this, but I
> noticed that I had Scipy import failures in for example optimize since
> the gfortran I used creates 32 bit code per default.
> 
> So I build a gcc 4.2.4 on OSX, then build python in 64 bit mode (fixing
> configure.in since Python assumes flags only available with the Apple
> gcc on Darwin), added a wrapper around gfortran that injects a "-m64"
> compile flag. Then I checkout out and build numpy (r5671), scipy (r4661)
> and the current nose release and ran the tests. I had some more failures
> than I expected (details below).
> 
> Any thoughts on the failures? I am about to build ATLAS and a couple
> other dependencies on that box.
> 
> William Stein is at Scipy 08 and there seems to be at least some
> interest in 64 bit OSX binaries. If anyone wants one let me know and I
> can put one together once Scipy 0.7 and Numpy 1.1.2 are out.
> 
> Cheers,
> 
> Michael
> 
> numpy failed tests:
> 
> ****************************************************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 29, in test_polynomial
> Failed example:
>      p / q
> Expected:
>      (poly1d([ 0.33333333]), poly1d([ 1.33333333,  2.66666667]))
> Got:
>      (poly1d([ 0.333]), poly1d([ 1.333,  2.667]))
> **********************************************************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 51, in test_polynomial
> Failed example:
>      p.integ()
> Expected:
>      poly1d([ 0.33333333,  1.        ,  3.        ,  0.        ])
> Got:
>      poly1d([ 0.333,  1.   ,  3.   ,  0.   ])
> **********************************************************************File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 53, in test_polynomialFailed example:
>      p.integ(1)
> Expected:
>      poly1d([ 0.33333333,  1.        ,  3.        ,  0.        ])
> Got:
>      poly1d([ 0.333,  1.   ,  3.   ,  0.   ])
> **********************************************************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 55, in test_polynomial
> Failed example:
>      p.integ(5)
> Expected:
>      poly1d([ 0.00039683,  0.00277778,  0.025     ,  0.        ,  0.
>      ,
>              0.        ,  0.        ,  0.        ])
> Got:
>      poly1d([ 0.   ,  0.003,  0.025,  0.   ,  0.   ,  0.   ,  0.   ,  0.
>    ])
> 
> ********************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 48, in test_ufunclike
> Failed example:
>      U.log2(a)
> Expected:
>      array([ 2.169925  ,  1.20163386,  2.70043972])
> Got:
>      array([ 2.17 ,  1.202,  2.7  ])
> **********************************************************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 53, in test_ufunclike
> Failed example:
>      U.log2(a, y)
> Expected:
>      array([ 2.169925  ,  1.20163386,  2.70043972])
> Got:
>      array([ 2.17 ,  1.202,  2.7  ])
> **********************************************************************
> File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 55, in test_ufunclike
> Failed example:
>      y
> Expected:
>      array([ 2.169925  ,  1.20163386,  2.70043972])
> Got:
>      array([ 2.17 ,  1.202,  2.7  ])
> ======================================================================
> FAIL: test_umath.TestComplexFunctions.test_it
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/nose/case.py",
> line 182, in runTest
>      self.test(*self.arg)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/core/test
> s/test_umath.py",
> line 196, in test_it
>      assert_almost_equal(fz.real, fr, err_msg='real part %s'%f)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py", 
> line 207, in assert_almost_equal
>      assert round(abs(desired - actual),decimal) == 0, msg
> AssertionError:
> Items are not equal: real part <ufunc 'log10'>
>   ACTUAL: -0.3010299956639812
>   DESIRED: 0.0
> 
> 
> scipy:
> 
> ======================================================================
> ERROR: test_nonsymmetric_modes (test_arpack.TestEigenNonSymmetric)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 204, in test_nonsymmetric_modes
>      self.eval_evec(m,typ,k,which)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 186, in eval_evec
>      eval,evec=eigen(a,k,which=which,**kwds)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/arpack.py",
> line 220, in eigen
>      raise RuntimeError("Error info=%d in arpack"%info)
> RuntimeError: Error info=-8 in arpack
> 
> ======================================================================
> ERROR: test_starting_vector (test_arpack.TestEigenNonSymmetric)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 214, in test_starting_vector
>      self.eval_evec(self.symmetric[0],typ,k,which='LM',v0=v0)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 186, in eval_evec
>      eval,evec=eigen(a,k,which=which,**kwds)
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/arpack.py",
> line 220, in eigen
>      raise RuntimeError("Error info=%d in arpack"%info)
> RuntimeError: Error info=-8 in arpack
> ======================================================================
> FAIL: test_implicit (test_odr.TestODR)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/odr/tests
> /test_odr.py", 
> line 118, in test_implicit
>      2.0778813389755596e-03]]),
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py", 
> line 304, in assert_array_almost_equal
>      header='Arrays are not almost equal')
>    File 
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py", 
> line 289, in assert_array_compare
>      assert cond, msg
> AssertionError:
> Arrays are not almost equal
> 
> (mismatch 4.0%)
>   x: array([[  2.10892628e+00,  -1.94376722e+00,   7.02635228e-02,
>           -4.71752493e-02,   5.25155371e-02],
>         [ -1.94376722e+00,   2.04814914e+00,  -6.16004809e-02,...
>   y: array([[  2.10892746e+00,  -1.94376864e+00,   7.02635509e-02,
>           -4.71752674e-02,   5.25155759e-02],
>         [ -1.94376864e+00,   2.04815092e+00,  -6.16005159e-02,...
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion



More information about the Numpy-discussion mailing list