[IPython-dev] ipcluster with clusterfile

Brian Granger ellisonbg.net@gmail....
Sun Nov 16 16:46:30 CST 2008


Vishal,

OK, I just commited the new ipcluster to my launcpad branch here:

https://code.launchpad.net/~ellisonbg/ipython/trunk-dev

Please have a look and hack away.  A few notes though:

* For now, we are assuming in ipcluster that all the nodes have a
shared filesystem.

* We are using Twisted for all the process management things.  Here
are the docs on that part of Twisted.

http://twistedmatrix.com/projects/core/documentation/howto/process.html

Let me know if you have questions about twisted.

* Currently the new version supports 3 cluster types (localhost, mpirun and pbs)

* We would love to have an ssh cluster as well and any help you want
to give in this area would be welcomed.

* We are planning on refactoring our config system.  This will change
many things in ipcluster, but for the better.  But, this won't happen
overnight.  I just wanted to give you the heads up that thingsa are
still changing a low.

* Some things in ipclsuter need to be refactored as well and tests
need to be added.

* I am not sure how soon these changes will be merged into trunk.  So,
for now I would work off my branch.

If you start hacking on the ssh cluster, keep in touch so we can coordinate.

Cheers,

Brian

On Thu, Nov 13, 2008 at 3:02 AM, Vishal Vatsa <vishal.vatsa@gmail.com> wrote:
> Hi Brian,
>
> just wondering if you had a chance to push the ipcluster code yet.
>
> Thanks,
> -vishal
>
> Brian Granger wrote:
>
>> OK, I will keep you posted and get you code ASAP.  It may be  few days
>> though as I am really busy on some other things through tomorrow.
>>
>> Brian
>>
>> On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa <vishal.vatsa@gmail.com>
>> wrote:
>>> Brian,
>>>
>>> Thanks for the reply.
>>>
>>> I am interested in ssh cluster. Mainly I would like to use
>>> the cluster file functionality to describe my cluster and
>>> startup the controller/engines.
>>>
>>> I am more than happy to be a tester/helper, let me know
>>> what I can do to help.
>>>
>>> -vishal
>>>
>>> Brian Granger wrote:
>>>
>>>> Vishal,
>>>>
>>>> Actually we have been working on ipcluster some.  What aspect of
>>>> ipcluster are you most interested in?  (localhost on Windows, ssh
>>>> cluster, PBS, etc)
>>>>
>>>> While our work is not done, I am more than willing to share the new
>>>> code with you (I just need to push it up to our bazaar repos on
>>>> launchpad) so you can help test it/work on it for your usage case.
>>>>
>>>> Cheers,
>>>>
>>>> Brian
>>>>
>>>> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa <vishal.vatsa@gmail.com>
>>>> wrote:
>>>>> Brian Granger wrote:
>>>>>
>>>>>>> We're at the RC1 level now, so I'd like to hold off on anything but
>>>>>>> really focused changes.  If you think this requires major work, I'd
>>>>>>> say let's disable it altogether and document it, so we can cut the
>>>>>>> release quickly.
>>>>>>
>>>>>> This was my leaning as well.  For this release, I am going to disable
>>>>>> the clusterfile and simply print a message to the user that it has
>>>>>> been disabled.
>>>>>>
>>>>>>> Once the release is out, we can open again a 'merge window' for more
>>>>>>> invasive changes.
>>>>>>>
>>>>> Guys,
>>>>>
>>>>> Is any one working on ipcluster?
>>>>>
>>>>> If not, could you explain the issues to me and I would
>>>>> love to have a go at the implementation.
>>>>>
>>>>>
>>>>> -vishal
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev@scipy.org
>>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev@scipy.org
>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>>>
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev@scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


More information about the IPython-dev mailing list