[Numpy-discussion] "import numpy" is slow

[Christopher Barker]

David Cournapeau wrote:
>> time python -c "import numpy"
>>
>> real    0m8.383s
>> user    0m0.320s
>> sys     0m7.805s

> What does python -c "import sys; print sys.path" say ?

A lot! 41 entries, and lot's of eggs -- are eggs an issue? I'm also 
wondering how the order is determined -- if it looked in site-packages 
first, it would find numpy a whole lot faster.

I also tried:

python -v -v -c "import numpy" &>junk2.txt

which results in:

# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# trying 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site.so
# trying 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/sitemodule.so
# trying 
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site.py

...
...

And a LOT more:

$ grep "# trying" junk2.txt | wc -l
     7446

For comaprison:
$ python -v -v -c "import sys" &>junk3.txt
$ grep "# trying" junk3.txt | wc -l
      618

which still seems like a lot.

So I think I've found the problem, it's looking in 7446 places ! but why?

I suspect the thing to do is to re-install from scratch, and only add in 
packages I'm really using now.

I wonder if making sure all eggs are unzipped would help, too.

Thanks for all your help on what is really OT at this point.

-Chris


$ python -c "import sys; [sys.stdout.write(p+'\n') for p in  sys.path]; 
print len(sys.path) "

/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/altgraph-0.6.7-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Pyrex-0.9.5.1a-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Shapely-1.0-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/ipython-0.8.2-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/py2app-0.4.2-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/modulegraph-0.7.2.dev_r21-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PyPubSub-3.0a5-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/macholib-1.2.1.dev_r23-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib-0.98.0-py2.5-macosx-10.3-fat.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.10.3-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Pylons-0.9.7beta5-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Tempita-0.2-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/WebError-0.8-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/WebOb-0.9.2-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Mako-0.2.0-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/decorator-2.2.0-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/simplejson-1.8.1-py2.5-macosx-10.3-ppc.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/FormEncode-1.0.1-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteScript-1.6.3-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PasteDeploy-1.3.2-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Paste-1.6-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Beaker-0.9.5-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/WebHelpers-0.6dev_20080613-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Routes-1.9-py2.5.egg
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/bdist_mpkg-0.4.3-py2.5.egg
/Users/cbarker/HAZMAT/Hazpy/trunk/lib
/usr/local/lib/wxPython-unicode-2.8.8.1/lib/python2.5/site-packages
/usr/local/lib/wxPython-unicode-2.8.8.1/lib/python2.5/site-packages/wx-2.8-mac-unicode
/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PIL
/usr/local/lib/wxPython-unicode-2.8.8.1/lib/python2.5
41




>> Any idea what could be wrong? I have no clue where to start, though I
>> suppose a complete clean out and re-install of python comes to mind.
>>
>> oh, and this is a dual G5 PPC (which should have a faster disk than your
>> Macbook)
> 
> disk should not matter. If hot, everything should be in the IO buffer,
> opening a file is of the order of a few micro seconds (that's
> certainly the order on Linux;  the VM on Mac OS X is likely not as
> good, but still).
> 
> cheers,
> 
> David
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov