[SciPy-user] NumPy vs. SciPy and other speed comparisons

Pearu Peterson pearu@cens.ioc...
Wed Jun 11 06:27:22 CDT 2008


Ok, found it. To fix the long scipy import time one needs to apply
Roberts patch to numpy/_import_tools.py and the following patch to
scipy:

Index: scipy/__init__.py
===================================================================
--- scipy/__init__.py   (revision 4424)
+++ scipy/__init__.py   (working copy)
@@ -31,9 +31,11 @@
  _num.seterr(all='ignore')

  __all__ += ['oldnumeric']+_num.__all__

  __all__ += ['randn', 'rand', 'fft', 'ifft']

+if 'linalg' in __all__:
+    __all__.remove('linalg')
+
  __doc__ += """
  Contents
  --------

With these patches, the 'from scipy import *' time drops from
0.6 to 0.2 in my computer.

Pearu


Pearu Peterson wrote:
> 
> Robert Kern wrote:
>> On Wed, Jun 11, 2008 at 05:33, Robert Kern <robert.kern@gmail.com> wrote:
>>> On Wed, Jun 11, 2008 at 05:06, Pearu Peterson <pearu@cens.ioc.ee> wrote:
>>>> We used to have postponed import hooks for that in past
>>>> but afaik we dropped these because they were hackish and at some moment
>>>> the time of importing scipy improved (I think it was in Python 2.4 or
>>>> 2.5) considerably.
>>> The problem is that we left scipy.pkgload in, and creating that from
>>> numpy._import_tools.PackageLoader implicitly adds all of the
>>> subpackages to scipy.__all__.
>> Correction: we actually do call pkgload(postpone=True). However, even
>> with the postponed import (which does not add any proxy objects), it
>> still appends to __all__. I believe the following patch to numpy fixes
>> the problem, but I'm not sure if leaving the if clause alone is
>> correct in all cases:
>>
>> Index: numpy/_import_tools.py
>> ===================================================================
>> --- numpy/_import_tools.py      (revision 5245)
>> +++ numpy/_import_tools.py      (working copy)
>> @@ -183,9 +183,6 @@
>>              postpone_import = getattr(info_module,'postpone_import',False)
>>              if (postpone and not global_symbols) \
>>                     or (postpone_import and postpone is not None):
>> -                self.log('__all__.append(%r)' % (package_name))
>> -                if '.' not in package_name:
>> -                    self.parent_export_names.append(package_name)
>>                  continue
>>
>>              old_object = frame.f_locals.get(package_name,None)
>>
>>
>> This can probably go into numpy 1.1.1 as a bugfix, so I don't think
>> it's critical to work around it in scipy/__init__.py.
> 
> The patch does not affect the 'from scipy import *' time much.
> In my computer it is around 0.62secs in both cases when having
> names in __all__ or not.
> 
> The long import seems to be due to scipy/linalg/iterative.py that
> imports scipy.sparse which takes most of the import time. And 
> scipy.linalg should not be imported when importing scipy.
> So, I think the real reason is hiding somewhere else.. I am looking into 
> it...
> 
> Pearu
> _______________________________________________
> SciPy-user mailing list
> SciPy-user@scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user


More information about the SciPy-user mailing list