[IPython-User] Asynchronous Plotting

Brian Granger ellisonbg@gmail....
Tue Dec 18 01:23:40 CST 2012


Michael,

Sorry no one has responded yet.  Here is a rough sketch of the issues involved.

In short, it is not possible to do what you want using threads in a
safe, reliable and robust manner.  IPython used to attempt to
integrate with GUI event loops using threads and it was horrible, both
from the code standpoint and the user experience.  But our built in
event loop support that uses a better approach should work fine out of
the box.  If you start ipython with:

ipython --pylab qt

You will get almost all the way there, but won't be able to interrupt
and computation in the foreground.

But if you use either the notebook or the qtconsole, you can still use
the qt pylab flag and you will be able to interrupt a calculation in
the foreground.  Why don't you give these a shot and let us know if
this works for you.

Cheers,

Brian

On Sat, Dec 15, 2012 at 11:59 AM, Michael McNeil Forbes
<michael.forbes@gmail.com> wrote:
> This question has been asked several times, but I can find no
> satisfactory answer:
>
> How can I have plots generated in the background while a calculation
> runs in the main thread (so that I can interrupt the calculation with
> Ctrl-C for example.)  My goals are:
>
> 1) Main calculation does not block when calling a "plot" function to
> plot data as it is computed.
> 2) Main calculation can be interrupted by user with Ctrl-C.
> 3) Plotting occurs in "background" plotting.
> 4) Cross-platform.  (At least on Mac OS X and Linux).
> 5) Simple.
>
> I envision a multithreaded approach (see below), but 2) seems to
> require that the calculation be in the main thread, which the plotting
> GUI's do not like.  A multiprocessing approach seems like it should
> also work (like http://stackoverflow.com/a/4662511/1088938), but on my
> Mac (10.5) I get
>
> The process has forked and you cannot use this CoreFoundation
> functionality safely. You MUST exec().
> Break on
> __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__
> () to debug.
>
> Also, I think that the GIL is probably good here so I do not have to
> copy data between processes, nor do I need to worry about the
> calculation thread updating the data while the plot thread is reading
> etc.
>
> I feel there must be some sort of simple solution with IPython, but I
> will be darned if I can figure it out.  It seems like the pre 1.11
> ipython solution of running the calculation in a secondary thread (by
> calling embed()) but I cannot figure out how to get user interrupts to
> affect that thread (point 2).
>
> Would something like twisted help?
>
> I know I could run two separate python instances, and use sockets to
> communicate back and forth, but this approach seems much more
> complicated (the user needs to start both processes for example).
> Maybe I could use parallelpython to automate this, but it still seems
> significantly more complicated than it should be.
>
> Any suggestions would be greatly appreciated.
>
> Here is what I envision would work if I could run the plotting in a
> secondary thread: The computational thread calls update() which pushes
> the arguments into a queue discarding unused previous values.  The
> plotting thread should get these from the queue and plot as quickly as
> it can.
>
> In fact, this seems to work on linux with ipython --pylab wx, but I
> can't figure out how to get this working on my Mac (the plot window
> never update).  It also fails with other backends (probably because I
> am trying to plot from a non-main thread.)
>
> Thanks,
> Michael.
>
> import time
> from threading import Thread, Event
> from Queue import Queue, Empty, Full
> import warnings
>
> #from multiprocessing import Process as Thread
> #from multiprocessing import Queue, Event
>
> from matplotlib import pyplot as plt
>
>
> class PylabPlot(Thread):
>      def __init__(self, timeout=60):
>          r"""
>          Parameters
>          ----------
>          timeout : float
>             Kill the thread if no activity for this many seconds.
>          """
>          self.timeout = timeout
>          self.queue = Queue(maxsize=1)
>          self.event = Event()
>          Thread.__init__(self)
>          self.dead = False
>          self.running = False
>          self.start()
>
>      def stop(self):
>          self.running = False
>          self.event.set()
>
>      def update(self, *v, **kw):
>          if self.dead:
>              warnings.warn("Plot thread is dead... ignoring update
> command.")
>              return
>          try:
>              self.queue.put((v, kw), block=False)
>          except Full:
>              # Remove an item, then put this one.
>              try:     # draw might be called in the meantine...
>                  self.queue.get(block=False)
>              except Empty:
>                  pass
>              self.queue.put((v, kw), block=False)  # This should not
> fail
>          self.event.set()
>
>      def run(self):
>          self.running = True
>          while self.running:
>              self.event.wait(timeout=self.timeout)
>              self.event.clear()
>              if self.queue.empty():
>                  self.running = False
>              else:
>                  while not self.queue.empty():
>                      try:
>                          v, kw = self.queue.get(block=False)
>                      except Empty:
>                          pass
>                      self.draw(*v, **kw)
>          self.dead = True
>
>
> def go():
>      import numpy as np
>
>      class Plot(PylabPlot):
>          def draw(self, x, y):
>              plt.ioff()
>              plt.clf()
>              plt.plot(x, y)
>              print y
>              plt.ion()
>              plt.draw()
>              time.sleep(0.5)     # Pretend this is slow
>
>      plt.clf()
>      tmp = raw_input("Position plot window, and press enter to run...")
>      x = np.linspace(0, 2*np.pi, 10)
>      dt = 0.001
>      t = 0
>      p = Plot()
>      try:
>          for n in xrange(10000):
>              t += dt
>              y = np.sin(x - t)
>              p.update(x=x, y=y)   # This should not take long and
> should not block
>      finally:
>          p.stop()   # Make sure that we exit gracefully.
>
> if __name__ == "__main__":
>      go()
> _______________________________________________
> IPython-User mailing list
> IPython-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-user



-- 
Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com


More information about the IPython-User mailing list