[SciPy-dev] mlabwrap scikit [Was: Re: scikits project]

Robert Kern robert.kern@gmail....
Thu Mar 29 21:05:36 CDT 2007

Alexander Schmolck wrote:
> Robert Kern <robert.kern@gmail.com> writes:
> Thanks for the feedback!
>>> I assume this question reflects my svn-newbie status, but why doesn't the
>>> scikits structure look something like this, given that as I understand it
>>> scikits are meant to be essentially independent (and hence independely
>>> versioned) projects under a common namespace?
>>>       mlabwrap/
>>>         branches/
>>>         tags/
>>>         trunk/
>>>           setup.py
>>>           scikits/
>>>             __init__.py
>>>             mlabwrap/
>>>               __init__.py
>>>       some_other_scikit/
>>>          branches/
>>>          ...
>>> It somehow seems to me that tags and branches should apply to individual
>>> projects and that one could still do convenient releases and svn-checkouts of
>>> scikits/ as a whole by using e.g. svn:externals to the individual projects.
>> branches/ and tags/ directories can be shared between projects. There's nothing
>> special about a branch or a tag in SVN; they are just copies. Just make sure you
>> name your branches and tags appropriately (mlabwrap-1.0, etc.). I would like all
>> of the scikits to be under one trunk, though, for conveniently grabbing the
>> current source of all of the scikits without also grabbing every branch and
>> tag.
> This is actually exactly what I thought svn:externals would solve (but maybe I
> don't understand the problems associated with that solution correctly):
>  mlabwrap/
>     branches/
>     tags/
>     trunk/
>       ...
>  some_other_scikit/
>     branches/
>     ...
>  scikits
>     trunk
>       scikits
>         -> mlabwrap/trunk/            # -> = external
>         -> some_other_scikit/trunk/
>         ...
>     tags
>         ...
> Thus, if you want all scikits, you just checkout above scikits sub-dir. (I
> presumably haven't got the directory-structure quite right, but I hope it
> doesn't matter for getting the point across)

> On the other hand, if the tags/ and branches/ directories are shared between
> all projects, you need to prefix all tags/branches with the project name
> (which would otherwise not be necessary) and have potentially quite a lot of
> dirs too look at that don't interest you in the least (e.g. when browsing
> tags/ or branches/ with trac), and, AFAICT, no particularly easy way to just
> check out everything related to your project. I assume this could also be
> fixed by 'symlinking' via externals, but you'd need many, many more (per
> project: one for the trunk and one for each branch and tag, vs. one for the
> trunk of each project -- and possibly the odd 'whole' scikits release tag).

By and large, it simply doesn't matter to "get everything related to your
project". Believe me, you don't want all of branches/ and tags/ in a checkout.
On the other hand, "getting all of the packages in scikits" does matter. IMO,
the inconvenience of prefixing your branches and tags is secondary to the
performance problems of svn:external, which slows down all checkouts and updates
 for everyone (although I'll have to double-check that claim for svn:external
links within the same repository).

Also, it appears that Trac doesn't like svn:external to other repositories; I'm
not sure if that extends to svn:external within the same repository.


> Maybe the structure I proposed would also make importing and exporting of
> projects slightly easier (because it mirrors the typical layout of an
> individual svn project). Speaking of which -- is there something I can do with
> the existing CVS so that it can be easily imported in the scikits svn (in
> which case we can get rid of what's already checked in), or would importing
> the CVS involve a hassle in any case, because then I'll just archive it on
> sourceforge.

I don't know. You'll have to read the cvs2svn documentation.

> The other thing I've been wondering is if such a setup couldn't also be made
> to accomodate something like Stefan van der Walt's layout proposal, which as
> far as I can see would allow for the most convenient way possible to grab all
> scikits and build them:
>        setup.py
>        scikits/
>          __init__.py
>          -> mlabwrap/
>              mlabwrap_setup.py
>              __init__.py
>              awmstools.py
>              ...
>          -> some_other_scikit/ 
>              some_other_scikit_setup.py

Having two ways to install something is just begging for trouble.

>            ...
> Couldn't one have a toplevel setup.py that just runs all
> scikits/DIRNAME/DIRNAME_setup.py's it can find, passing through the command line
> options (or something along those lines[1])?

That's unworkable. I've tried.

>> If so it should also use numpy.distutils. Just make sure to import
>> setuptools first.
>>   import setuptools
>>   from numpy.distutils.core import setup
>>   ...
> Is there a recipe/template for this somewhere? Googling "scipy setuptools"
> comes up with
>  <http://projects.scipy.org/ipython/ipython/wiki/SetupTools>
> as the first hit, which seems to indicate that setuptools is still a bit alpha
> and the docs can't be trusted if one wants something that actually works.

Fernando's a curmudgeon, and that page is old. Ignore him.  :-)

Like I said, just import setuptools before you import numpy.distutils. Then use
numpy.distutils as normal to handle all of the building and stuff. setuptools
adds some keywords to setup() that you should also provide, namely


That's all that's necessary. There's no particular magic to combining setuptools
and numpy.distutils. Read the setuptools documentation for other keyword
arguments the setup() that you might want to use for the extra features it
provides, like install_requires.

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

More information about the Scipy-dev mailing list