[IPython-User] Is iPython useful for this scenario?
Mon Feb 6 13:28:30 CST 2012
On Sat, Feb 4, 2012 at 2:03 PM, Florian Lindner <email@example.com> wrote:
> I'm currently working on a control-/queue-management software for a
> CFD simulation system. If consists of three parts:
> - A client communicates with the CFD system. It can be long running.
> The client can also be run standalone.
> - A server which does the queue management and starts up the clients.
> It is non-interactive
> - A server interface which the user uses to talk to the server, e.g.
> to enqueue new jobs.
I am following your design here, but the naming of things is a bit
backwards from IPython. Here is our terminology:
* Engine = runs on a compute node and does the actual computation.
This is where the CFD sim would run.
* Controller = Schedules tasks to engines using lightweight, low
* Cluster = Starts Engines/Controller using batch system.
* Client = Frontend process that the users uses to talk to the above.
> Currently they are communicating via XMLRPC (from python stdlib):
> client <---- server <---- server interface.
This architecture is partially reinventing everything in
IPython.parallel. I would just use IPython.parallel and take
advantage of everything we have there. It is extremely powerful and
will out perform XMLRPC by a long shot.
> A this time the system works only localhost and with one client.
IPython supports a wide range of cluster configurations (PBS, Torque,
mpiexec, SSH, etc.) and multiple engines and clients.
> Before continue to extend it I wonder if iPython could be useful for
> network communication and process management. I browsed through the
> docs but I'm not entirely sure if I got the ideas of iPython right.
> The user should not get in contact with iPython. The software is not
> doing and probably will never do any numerical demanding calculations
> Is iPython useful in the scenario?
It would be extremely useful. I would check out our cluster docs here:
The notebook would also be useful as well:
> IPython-User mailing list
Brian E. Granger
Cal Poly State University, San Luis Obispo
firstname.lastname@example.org and email@example.com
More information about the IPython-User