[IPython-User] Asynchronous Plotting

[Michael Forbes]

This question has been asked several times, but I can find no satisfactory
answer:

How can I have plots generated in the background while a calculation runs
in the main thread (so that I can interrupt the calculation with Ctrl-C for
example.)  My goals are:

1) Main calculation does not block when calling a "plot" function to plot
data as it is computed.
2) Main calculation can be interrupted by user with Ctrl-C.
3) Plotting occurs in "background" plotting.
4) Cross-platform.  (At least on Mac OS X and Linux).
5) Simple.

I envision a multithreaded approach (see below), but 2) seems to require
that the calculation be in the main thread, which the plotting GUI's do not
like.  A multiprocessing approach seems like it should also work (like
http://stackoverflow.com/a/4662511/1088938), but on my Mac (10.5) I get

The process has forked and you cannot use this CoreFoundation functionality
safely. You MUST exec().
Break on
__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__()
to debug.

Also, I think that the GIL is probably good here so I do not have to copy
data between processes, nor do I need to worry about the calculation thread
updating the data while the plot thread is reading etc.

I feel there must be some sort of simple solution with IPython, but I will
be darned if I can figure it out.  It seems like the pre 1.11 ipython
solution of running the calculation in a secondary thread (by calling
embed()) but I cannot figure out how to get user interrupts to affect that
thread (point 2).

Would something like twisted help?

I know I could run two separate python instances, and use sockets to
communicate back and forth, but this approach seems much more complicated
(the user needs to start both processes for example).  Maybe I could use
parallelpython to automate this, but it still seems significantly more
complicated than it should be.

Any suggestions would be greatly appreciated.

Here is what I envision would work if I could run the plotting in a
secondary thread: The computational thread calls update() which pushes the
arguments into a queue discarding unused previous values.  The plotting
thread should get these from the queue and plot as quickly as it can.

In fact, this seems to work on linux with ipython --pylab wx, but I can't
figure out how to get this working on my Mac (the plot window never
update).  It also fails with other backends (probably because I am trying
to plot from a non-main thread.)

Thanks,
Michael.


import time
from threading import Thread, Event
from Queue import Queue, Empty, Full
import warnings

#from multiprocessing import Process as Thread
#from multiprocessing import Queue, Event

from matplotlib import pyplot as plt


class PylabPlot(Thread):
   def __init__(self, timeout=60):
       r"""
       Parameters
       ----------
       timeout : float
          Kill the thread if no activity for this many seconds.
       """
       self.timeout = timeout
       self.queue = Queue(maxsize=1)
       self.event = Event()
       Thread.__init__(self)
       self.dead = False
       self.running = False
       self.start()

   def stop(self):
       self.running = False
       self.event.set()

   def update(self, *v, **kw):
       if self.dead:
           warnings.warn("Plot thread is dead... ignoring update command.")
           return
       try:
           self.queue.put((v, kw), block=False)
       except Full:
           # Remove an item, then put this one.
           try:     # draw might be called in the meantine...
               self.queue.get(block=False)
           except Empty:
               pass
           self.queue.put((v, kw), block=False)  # This should not fail
       self.event.set()

   def run(self):
       self.running = True
       while self.running:
           self.event.wait(timeout=self.timeout)
           self.event.clear()
           if self.queue.empty():
               self.running = False
           else:
               while not self.queue.empty():
                   try:
                       v, kw = self.queue.get(block=False)
                   except Empty:
                       pass
                   self.draw(*v, **kw)
       self.dead = True


def go():
   import numpy as np

   class Plot(PylabPlot):
       def draw(self, x, y):
           plt.ioff()
           plt.clf()
           plt.plot(x, y)
           print y
           plt.ion()
           plt.draw()
           time.sleep(0.5)     # Pretend this is slow

   plt.clf()
   tmp = raw_input("Position plot window, and press enter to run...")
   x = np.linspace(0, 2*np.pi, 10)
   dt = 0.001
   t = 0
   p = Plot()
   try:
       for n in xrange(10000):
           t += dt
           y = np.sin(x - t)
           p.update(x=x, y=y)   # This should not take long and should not
block
   finally:
       p.stop()   # Make sure that we exit gracefully.

if __name__ == "__main__":
   go()

P.S. Sorry if this post appears twice: I have been having some problems
with my mail server.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/ipython-user/attachments/20121215/539424a2/attachment.html