[Numpy-discussion] Going toward time-based release ?

Jarrod Millman millman@berkeley....
Mon May 12 02:28:47 CDT 2008

On Mon, May 12, 2008 at 12:09 AM, Stéfan van der Walt <stefan@sun.ac.za> wrote:
> Which brings me to my next point: we have very limited (human)
> resources, but releasing frequently is paramount.  To what extent can
> we automate the release process?  I've asked this question before, but
> I haven't had a clear answer: are the packages currently built
> automatically?  Why don't we get the buildbots to produce nightly
> snapshot packages, so that when we tell users "try the latest SVN
> version, it's been fixed there" it doesn't send them into a dark
> depression?

The packages aren't built automatically.  We just changed the package
build process.  As of now David C. is building the Windows binaries
and Chris B. is building the Mac OS X binaries.  In terms of getting
the releases out, this is not a very important problem.  The main
issue is getting the code stable enough to make a release.  However,
your suggestion to have nightly binaries autogenerated would be more
useful than telling users to try the latest SVN.

I assume that we should be able to have these binaries auto-generated,
David and Chris should be able to provide a more detailed answer to
this than I can.  However, my concern with this is that, I believe, we
currently need some oversight to ensure that the binaries don't have
silly problems.

Regardless of whether we automate the process or still have manual
aspect to the process, it is a good idea to start creating binaries
for release candidates or test releases.

> Memory errors:  Albert Strasheim recently changed his build client
> config to run Valgrind on the NumPy code.  Surprise, surprise -- we
> introduced new memory errors since the last release.  In the future,
> when *any* changes are made to the the C code:
> a) Add a unit test for the change, unless the test already exists (and
> I suggest we *strongly* enforce this policy).
> b) Document your change if it is not immediately clear what it does.
> b) Run the test suite through Valgrind, or if you're not on a linux
> platform, look at the buildbot (http://buildbot.scipy.org) output.

I would be happy to see a policy like this adopted.  You have my vote.

> Finally, our patch acceptance process is poor.  It would be good if we
> could have a more formal system for reviewing incoming and *also* our
> own patches.  I know Ondrej Certik had a review board in place for
> Sympy at some stage, so we could ask him what their experience was.

We need to be very careful not to make the process for contributing
code too burdensome.  The number of developers is increasing and our
development community is growing; I don't want to see this process

If we go this route, we may need better tools for creating and
reviewing patches.  I would recommend taking it slow.  There are a
number of suggestions for improving our development process.  I would
prefer changing only one or just a few aspects at a time.  That way we
can more easily understand what effect these changes have on our
development process.

This is another argument for increasing the frequency of releases.  It
would give us more of an opportunity to review and improve the

Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014

More information about the Numpy-discussion mailing list