[Nipy-devel] [nipype] better cluster support proposal

Satrajit Ghosh satra@mit....
Thu Jan 27 14:35:59 CST 2011


thanks michael,


> Simply the "interactive" aspects of IPython. It needs a low-latency
> scheduler to be interactive. OTOH it needs to make compromises to get
> low-latency that effect the throughput of a compute-cluster (we had a
> short exchange on ipython-user on that). I've put Brian in touch with
> the Condor group that expressed their interest in this use-case -- we'll
> see what will emerge from this...
>

we want the interactive aspects in nipype, right? when a workflow is
running, i should be able to monitor which nodes have executed, which are
executing, query the outputs of the nodes, etc.,. which will require
communication with the scheduler (ipython or not)

if we created a version of nipype with python-drmaa instead, would that help
your case? i believe condor and sge both support drmaa.


> > (regarding ipython, the ability to ship an entire dag to ipython will be
> > available in 0.11)
>
> Cool!
>
>
> Thanks,
>
> Michael
>
>
> --
> Michael Hanke
> http://mih.voxindeserto.de
> _______________________________________________
> Nipy-devel mailing list
> Nipy-devel@neuroimaging.scipy.org
> http://mail.scipy.org/mailman/listinfo/nipy-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/nipy-devel/attachments/20110127/2e3a2938/attachment.html 


More information about the Nipy-devel mailing list