[Numpy-discussion] A memory problem: why does mmap come up in numpy.inner?

Michael Abshoff mabshoff@googlemail....
Wed Jun 4 22:59:09 CDT 2008


Dan Yamins wrote:
>
>
Hello folks,

I did port Sage and hence Python with numpy and scipy to 64 bit OSX and 
below are some sample build instructions for just building python and 
numpy in 64 bit mode.
>
>
>     Try
>
>     In [3]: numpy.dtype(numpy.uintp).itemsize
>     Out[3]: 4
>
>     which is the size in bytes of the integer needed to hold a
>     pointer. The output above is for 32 bit python/numpy.
>
>     Chuck
>
>
> Check, the answer is 4, as you got for the 32-bit.   What would the 
> answer be on a 64-bit architecture?  Why is this diagnostic?
>
> Thanks!
> Dan
>
First try:

export OPT="-g -fwrapv -O3 -m64 -Wall -Wstrict-prototypes"
./configure --disable-toolbox-glue 
--prefix=/Users/mabshoff/64bitnumpy/python-2.5.2-bin  

<SNIP>
checking for int... yes
checking size of int... 4
checking for long... yes
checking size of long... 4
<SNIP>

Oops, make fail because of the above. Let's try again:

 ./configure --disable-toolbox-glue 
--prefix=/Users/mabshoff/64bitnumpy/python-2.5.2-bin --with-gcc="gcc -m64"

<SNIP>
checking for int... yes
checking size of int... 4
checking for long... yes
checking size of long... 8
<SNIP>

make && make install

then:

bsd:python-2.5.2-bin mabshoff$ file bin/python
bin/python: Mach-O 64-bit executable x86_64

Let's make the 64 bit python default:

bsd:64bitnumpy mabshoff$ export 
PATH=/Users/mabshoff/64bitnumpy/python-2.5.2-bin/bin/:$PATH
bsd:64bitnumpy mabshoff$ which python
/Users/mabshoff/64bitnumpy/python-2.5.2-bin/bin//python
bsd:64bitnumpy mabshoff$ file `which python`
/Users/mabshoff/64bitnumpy/python-2.5.2-bin/bin//python: Mach-O 64-bit 
executable x86_64

Let's build numpy 1.1.0:

bsd:64bitnumpy mabshoff$ tar xf numpy-1.1.0.tar.gz
bsd:64bitnumpy mabshoff$ cd numpy-1.1.0
bsd:numpy-1.1.0 mabshoff$ python setup.py install
<SNIP>

bsd:python-2.5.2-bin mabshoff$ python
Python 2.5.2 (r252:60911, Jun  4 2008, 20:47:16)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy
 >>> numpy.dtype(numpy.uintp).itemsize
8
 >>> ^D
bsd:python-2.5.2-bin mabshoff$  

Voila ;)

Wi th numpy 1.0.4svn from 20080104 the numpy setup did not work since a 
conftest failed. I did report that to Stefan V. in IRC via #sage-devel 
and I also thought that this was still a problem with numpy 1.1.0, but 
fortuantely that was fixed.

Now on to the test suite:

 >>> numpy.test()
Numpy is installed in 
/Users/mabshoff/64bitnumpy/python-2.5.2-bin/lib/python2.5/site-packages/numpy
Numpy version 1.1.0
Python version 2.5.2 (r252:60911, Jun  4 2008, 20:47:16) [GCC 4.0.1 
(Apple Inc. build 5465)]
  Found 18/18 tests for numpy.core.tests.test_defmatrix
  Found 3/3 tests for numpy.core.tests.test_errstate
  Found 3/3 tests for numpy.core.tests.test_memmap
  Found 286/286 tests for numpy.core.tests.test_multiarray
  Found 70/70 tests for numpy.core.tests.test_numeric
  Found 36/36 tests for numpy.core.tests.test_numerictypes
  Found 12/12 tests for numpy.core.tests.test_records
  Found 143/143 tests for numpy.core.tests.test_regression
  Found 7/7 tests for numpy.core.tests.test_scalarmath
  Found 2/2 tests for numpy.core.tests.test_ufunc
  Found 16/16 tests for numpy.core.tests.test_umath
  Found 63/63 tests for numpy.core.tests.test_unicode
  Found 4/4 tests for numpy.distutils.tests.test_fcompiler_gnu
  Found 5/5 tests for numpy.distutils.tests.test_misc_util
  Found 2/2 tests for numpy.fft.tests.test_fftpack
  Found 3/3 tests for numpy.fft.tests.test_helper
  Found 24/24 tests for numpy.lib.tests.test__datasource
  Found 10/10 tests for numpy.lib.tests.test_arraysetops
  Found 1/1 tests for numpy.lib.tests.test_financial
  Found 53/53 tests for numpy.lib.tests.test_function_base
  Found 5/5 tests for numpy.lib.tests.test_getlimits
  Found 6/6 tests for numpy.lib.tests.test_index_tricks
  Found 15/15 tests for numpy.lib.tests.test_io
  Found 1/1 tests for numpy.lib.tests.test_machar
  Found 4/4 tests for numpy.lib.tests.test_polynomial
  Found 1/1 tests for numpy.lib.tests.test_regression
  Found 49/49 tests for numpy.lib.tests.test_shape_base
  Found 15/15 tests for numpy.lib.tests.test_twodim_base
  Found 43/43 tests for numpy.lib.tests.test_type_check
  Found 1/1 tests for numpy.lib.tests.test_ufunclike
  Found 89/89 tests for numpy.linalg.tests.test_linalg
  Found 3/3 tests for numpy.linalg.tests.test_regression
  Found 94/94 tests for numpy.ma.tests.test_core
  Found 15/15 tests for numpy.ma.tests.test_extras
  Found 17/17 tests for numpy.ma.tests.test_mrecords
  Found 36/36 tests for numpy.ma.tests.test_old_ma
  Found 4/4 tests for numpy.ma.tests.test_subclassing
  Found 7/7 tests for numpy.tests.test_random
  Found 16/16 tests for numpy.testing.tests.test_utils
  Found 5/5 tests for numpy.tests.test_ctypeslib
......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ctypes 
is not available on this python: skipping the test (import error was: 
ctypes is not available.)
.....
----------------------------------------------------------------------
Ran 1275 tests in 1.420s

OK
<unittest._TextTestResult run=1275 errors=0 failures=0>

I did also fixe the ctypes build in 64 bit OSX, but that is right now a 
truly horrible hack. Sage needs it for twisted, so it isn't an option. I 
will clean up my fix and post it to the python list soon [I have been 
busy with $WORK, so no clean version yet], but if anybody cares here I 
can post a simple step by step workaround. The issue here is that the 
libffi used by ctypes is way out of date and as is does not have 64 bit 
OSX support.

Hope this helps.

Congratulations on numpy 1.1.0 - I mostly lurk here, so I didn't have a 
chance yet.

Cheers,

Michael

> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>   




More information about the Numpy-discussion mailing list