[AstroPy] Co-ordinating Python astronomy libraries?

Perry Greenfield perry@stsci....
Thu Jul 8 09:13:05 CDT 2010


On Jul 6, 2010, at 11:15 PM, Thomas Robitaille wrote:
>
> This is a very good idea - it is similar to what I was thinking of  
> when I mentioned a hierarchical setup, but I went too far because -  
> as you pointed out - a full Python distribution would be a nightmare  
> to maintain. What you suggest is something more manageable, and in  
> the end encourages better practices (and after all, who needs yet  
> another full python distribution on their machine, besides e.g. the  
> system python, scisoft, macports, etc.!)
>
> Is there a standard way of combining separate packages, potentially  
> developed in different repositories, into a single meta-package for  
> distribution to the user? Does distutils or distribute allow  
> something like this by default?

Not reliably. Nor does easy_install work for many cases (the Python  
community is well aware of the current shortcomings of easy_install,  
(e.g., see: http://www.b-list.org/weblog/2008/dec/14/packaging/)

These often don't handle fortran code well (scipy uses a modified  
distutils  to handle those issues).

To really ensure that everything works basically requires bundling the  
tricky stuff (particularly non-python library dependencies) into the  
core (e.g., not relying on it being present on the host system) and  
testing it routinely to make sure it works on the platforms you want  
it to work on. I don't think there is a purely modular system that  
works reliably. I can envision a lot of modular add-ons that don't   
present a problem (for example, pure python code that has no new  
library dependencies, or python/C code that uses distutils in a simple  
and  reliable way). But there is a lot of stuff out there that doesn't  
fit that mode.

So to elaborate a bit more on what we are thinking about is making a  
core distribution that contains things that are generally useful to  
most astronomers, things required for our software ("our" currently  
being Gemini and STScI, but that could grow), or libraries that are  
hard to install later but are used by possible modular add ons (VTK is  
a good example). Then there would be  optional packages one  can add  
to that list. The core system and optional packages would be regularly  
built and tested on many platforms. Finally, there  would be  
instructions on how to install arbitrary software into that  
environment (not necessarily trivial work). While anyone will be able  
to update or add software to the environment (there are often good  
reasons to do so), given that arbitrary updates or additions could  
break other things in the environment, no guarantees on whether such  
things will work can be made (but you are free to do so). There is no  
perfect solution out there, or even one  close to perfect. The idea is  
to make installing and using the basic tools easy, and possible to  
customize that (but with risk that must be borne by the user). The  
most flexible approach is to install everything yourself, piece by  
piece, each thing in its own way. If you have special needs, sometimes  
that is the only real solution.

Perry



More information about the AstroPy mailing list