[Numpy-discussion] Numpy/Scipy for EC2

Dan Yamins dyamins@gmail....
Sat Dec 5 11:03:50 CST 2009


On Sat, Dec 5, 2009 at 1:32 AM, David Goldsmith <d.l.goldsmith@gmail.com>wrote:

> Dan-
>
>     I almost hate to ask - after what you've already provided, which is
> substantial: did you ever try out what alestic had to offer?  If not, I may
> be that guinea pig. ;-)
>
>
David -- I'm posting my response to your question to the group as well ...


I never tried out the alestic AMIs *directly* ... that's because at least as
far as I could tell (I might be wrong about this), they didn't have blas,
lapack, scipy, etc prebuilt on them.

But the whole StarCluster thing *is* basically Alestic fundamentally, since
the AMIs that StarCluster provides were themselves built directly from
Alestic's 32-bit and 64-bit Ubuntu jaunty AMIs ... with comparatively small
but crucial modifications (building optimized blas, scipy, the Sun Grid
Engine, NFS &c).  From a "systems" point of view should basically have the
same stability properties.

The real strength of starcluster is that, in addition to those small but
crucial additions to the Alestic AMIs, Starcluster also provides a simple
command for making startup/admin/termination of a multi-node cluster really
easy.  With one invocation, it will interface with EC2 commands to start up
the relevant machine instances, mount a shared NFS drive from an amazon EBS
volume, start up the grid engine, etc... so that all the headaches of
setting up and breaking down the pieces of a multi-node shared-memory
cluster are completely handled for you.

Is there a particular reason you had in mind for using Alestic directly?
I'd be interested to hear your use case ...

Dan

Btw -- I got an email from Justin Riley, the developer of starcluster -- he
seems to be hard at work  on the next release (which will have hadoop,
couchdb, and some important improvements to the management of multiple
clusters.  still no word on dynamic load balancing tho)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20091205/66f31095/attachment.html 


More information about the NumPy-Discussion mailing list