[SciPy-dev] Issues "freezing" scipy using py2app

Pearu Peterson pearu at scipy.org
Wed Dec 21 00:29:09 CST 2005



On Wed, 21 Dec 2005, Josh Marshall wrote:

> I have an application that is using PyObjC, matplotlib and scipy to
> perform some image classification and feature extraction. I am now
> attempting to use py2app to bundle it up into a single OS X .app.
>
> This has led to a few questions regarding the architecture of scipy,
> since there are a few problems I have had to work around.
>
> 1) The first concerns __init.py__ and _import_utils.py. In
> get_info_modules() from _import_tools, the presence of info.py files
> is required. However, when distributing a condensed version, these
> files are not included while the compiled info.pyc is. I have made a
> few alterations to _import_tools.py, such that if the info.py-s don't
> exist, then the .pyc files are tried instead. What I have done is the
> following:
>
> +      use_pyc = False
>         if packages is None:
>             info_files = glob(os.path.join
> (self.parent_path,'*','info.py'))
> +           info_files = [f for f in info_files if os.path.exists(f)]
> +            if not info_files:
> +                use_pyc =True
> +                info_files = glob(os.path.join
> (self.parent_path,'*','info.pyc'))
>
> and then, when we try actually importing them using load_module:
>
>         for info_file in info_files:
>             package_name = os.path.basename(os.path.dirname(info_file))
>             fullname = self.parent_name +'.'+ package_name
>
>             if use_pyc:
>                 filedescriptor = ('.pyc','rb',2)
>             else:
>                 filedescriptor = ('.py','U',1)
>
>             try:
>                 info_module = imp.load_module(fullname+'.info',
>                                               open
> (info_file,filedescriptor[1]),
>                                                 info_file,
>                                               filedescriptor)

Thanks for the patch. I have applied it to new pkgload feature. scipy does 
not use _import_tools.py anymore.

> There must be a simpler way, such as using find_module, but I
> couldn't get this to work since it requires the parent package to be
> imported before the subpackages.

info.py files should be regarded as information files about packages, they 
are python modules only for convinience. And using imp.load_module is 
necessary to prevent importing the whole package, at the import time we 
need only the information from the info.py file.

> 2) The second issue was a problem involving get_config_vars being
> called from ppimport.py. distutils then requires the presence of the
> Python config file, which doesn't exist in the bundle.
>
>  This was easily fixed by adding
>
>         if sys.platform[:5]=='linux':
>             so_ext = '.so'
> +        elif sys.platform=='darwin':
> +            so_ext = '.so'

Hmm, what exception do you get when Python config file is not present?
ppimport._get_so_ext uses so_ext='.so' when ImportError appears.
I think the proper fix would be to catch proper exception.

> 3) Finally, I can't distribute my .app as-is, after being generated
> by py2app. py2app operates by all needed modules from the system site-
> packages being collected into a "site-packages.zip" within the
> application bundle. At runtime, some magic with import causes
> everything to work, and this is where scipy falls down. When trying
> to import the subpackages using the _import_utils.py magic, it cannot
> find them within the site-packages.zip. I can fix this by
> distributing the .app with the site-packages.zip expanded into a
> directory. However, I was wondering if anyone who knows the import
> magic better knows if we can fix this.

I'd like to learn more about the magic that Python does with .zip files.
Are there similar issues when using eggs?

Pearu




More information about the Scipy-dev mailing list