[Numpy-discussion] Numpy 1.3.0 rc1 OS X Installer

[Christopher Barker]

David Cournapeau wrote:
> Christopher Barker wrote:
>> It does, but we don't need a binary installer for a python that doesn't 
>> have a binary installer.
> 
> Yes, not now - but I would prefer avoiding to have to change the process
> again when time comes. It may not look like it, but enabling a working
> process which works well on all platforms including windows took me
> several days to work properly.

I'm not surprised it took that long -- that sounds short to me!

Anyway, If there are new python builds that are 64-bit (quad?) you wont' 
have to change much -- "only" make sure that the libs you are linking to 
are 64 bit. I suppose you could try to get a quad-universal gfortran.a 
now, but I'd wait 'till you need it.

>> However, I'm not sure you need to do what your saying here. I imagine 
>> this workflow:
>>
>> set up a virtualenv for, say numpy x.y.rc-z
>>
>> play around with it, get everything to build, etc. with plain old 
>> setup.py build, setup.py install, etc.
>>
>> Once you are happy, run:
>>
>> /Library/Frameworks/Python.framework/Versions/2.5/bin/bdist_mpkg
>>   
> 
> This means building the same thing twice.

does it? I just did test:

I set up a virtualenv with nothing in it.

I used that environment to do:

setup.py build
setup.py install

and got an apparently working numpy in my virtualenv.

Then I ran:

/Library/Frameworks/Python.framework/Versions/2.5/bin/bdist_mpkg

It did a lot, but I don't THINK is re-compiled everthing. I think the 
trick is that eveything it built was still in the build dir -- and it is 
the saem python, even though it's not living in the same place.

I got what seems to be a functioning Universal Installer for the 
python.org python.


Having said that, it looks like it may be very easy to hack a package 
built in the virtualenv to install in the right place. In:

dist/numpy-1.3.0rc1-py2.5-macosx10.4.mpkg/Contents/Packages/

there are two mpkgs. IN each of those, there is:

Contents/Info.plist

which is an xml file. In there, there is:

<key>IFPkgFlagDefaultLocation</key>

which should be set to:

<string>/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages</string>

If it was built in a virtualenv, that would be the virtualenv path.

It's a hack, but you could post-process the mpkg to change that value 
after building.

> The docs are included in the .dmg, and yes, the doc needs to be built
> from the same installation (or more exactly the same source).

In my tests, that seems to work fine.

>> True. In that case we could put the dylib somewhere obscure:
>>
>> /usr/local/lib/scipy1.6/lib/
>>   
> Hm, that's strange - why /usr/local/lib ? It is outside the scipy
> installation.

I'm not sure, really, except that that's where Robin Dunn puts stuff for 
wxPython -- I think one reason may be that he can then point to it from 
more than one Python installation -- for instance Apple's and 
python.orgs. And it could help with building from a virtualenv.

>> or even:
>>
>> /Library/Frameworks/Python.framework/Versions/2.5/lib/
>>   
> That's potentially dangerous: since this directory is likely to be in
> LIBDIR, it means libgfortran will be taken there or from /usr/local/lib
> if the user builds numpy/scipy after installing numpy. If it is
> incompatible with the user gfortran, it will lead to weird issues, hard
> to debug.

showing what a pain all of this is! Of course, you could put it in:

/Library/Frameworks/Python.framework/Versions/2.5/lib/site_packages/scipy/lib/

or something that only scipy would know about.


> If we install something like libgfortran, it should be installed
> privately - but dynamically linking against private libraries is hard,
> because that's very platform dependent

yup -- on the Mac, it could work well, 'cause the paths to the libs are 
hard-coded when linked, so you WILL get the right one -- if it's there! 
Tough this could lead to trouble when you want to build from a 
virtualenv, and install to the system location. mocaholib gives you 
tools to re-write the locations, but someone would have to write that code.

> gfortran hello.f -> a.out is 8 kb
> gfortran hello.f -static-libgfortran -> a.out is 130 kb
> libgfortran.a -> ~ 1.3Mb


> And thinking about it: mac os x rather encourage big binaries - "fat
> binary", so I am not sure it is a big concern.

so I guess we're back to static linking -- I guess that's why it's 
generally "recommended" for distributing binaries.

>> Also, would it break anything if the libgfortran installed were properly 
>> versioned:
>>   libgfortran.a.b.c
> 
> versioned libraries only make sense for shared libraries,I think.

right -- I meant for the dynamic lib option.

> On
> Linux, the static library is not even publicly available (it is in
> /usr/lib/gcc/4.3.3). I wonder whether the mac os x gfortran binary did
> not make a mistake there, actually.

where are you getting that? I'm about to go on vacation, but maybe I 
could try a few things...

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov