[IPython-User] IPython cluster with PBS when only head node is accessible

Martin Luessi mluessi@gmail....
Tue Oct 22 09:51:01 CDT 2013


Hi,

I have been using IPython cluster in SSH mode for a while and it works
perfectly. It allows me to distribute jobs to a bunch of workstations
that would otherwise sit idle during the night :).

Anyways, I now have a larger job and I would like to run it on a
"proper cluster". We use a PBS system and I have configured an IPython
profile as described in

http://ipython.org/ipython-doc/dev/parallel/parallel_process.html#using-ipcluster-in-pbs-mode

I can successfully launch the cluster and engines from the head node
of the cluster. However, the problem is that the nodes of the cluster
are only accessible through the head node, i.e., I can't make a direct
SSH connection from my workstation (where IPython notebook is running)
to one of the nodes. Is there a way to still use the IPython cluster
in this case without making changes to the network setup? This issue
has been raised on stackoverflow:

http://stackoverflow.com/questions/5186989/parallel-ipython-ipcluster-through-head-node

but it seems there was no solution, so I thought I would ask here.

Thanks,

Martin


More information about the IPython-User mailing list