[IPython-dev] Project idea: Automatic lab notebook for iPython
Wed Apr 17 08:59:54 CDT 2013
I think this is the kind of question that a lot of us are interested in,
and I'd be keen to see support for this integrated with IPython notebooks.
I won't have much time to help with this for a few months, but I'll give a
brief run down of the architecture, so you can see where your tools might
- The notebook server is an HTTP server - at present, most users run it
locally, but multi-user servers are definitely planned.
- When you open a notebook in the browser, the server starts a
corresponding 'kernel', a Python interpreter communicating over ZMQ.
- As you interact with the notebook, messages like 'execute this code' and
'get tab completions for this line' go from the browser to the kernel, and
replies come back.
the current state of your input and output cells, and sends it to the
server as JSON. The server dumps it to a file, or in the future to a
We'll be happy to answer any questions you have.
On 17 April 2013 00:38, Peter Macko <firstname.lastname@example.org> wrote:
> Hi iPython developers,
> Here is a new project idea: automatic lab notebook for iPython and
> iPython Notebook, which would keep track of how each of your output
> files was produced, linking this "history" (or a "lineage") of an object
> across different iPython sessions and different iPython notebooks, and
> storing it persistently. This is frequently referred to in the Computer
> Science literature as "provenance."
> It will enable you to ask questions like "what did I do to produce this
> plot?" - and for example, it will tell you that you downloaded the input
> data set on Monday from such and such website, you ran all these
> commands to process the data on Tuesday, and then produced this plot on
> Thursday from a different iPython session. Note that this goes beyond
> (and is complementary in purpose to) iPython Notebook, since the history
> of a file is tracked across different sessions and Notebooks, and when
> you ask a question, you will get only the relevant information,
> suppressing any additional things that you did that are unrelated to the
> file in which you are interested.
> We are in touch with computational scientists all the way from
> bioinformatics to physics that are very interested in this feature! We
> met their needs partially by developing a cross-platform, multi-lingual
> library (https://code.google.com/p/core-provenance-library/) that they
> can use to annotate their Python (and non-Python) scripts in order to
> track the lineage of their objects.
> Our vision is that this will be all done fully automatically, without
> requiring the users to manually annotate their scripts. But
> unfortunately neither of us who are involved in this project has the
> resources or the knowledge of the iPython code-base to tackle this
> challenge. We need your help to make this happen! We have some ideas
> about how we might go about this, but we need someone who knows more
> about iPython to talk them over and to spearhead the actual development.
> Please let us know if you can help!
> Thank you,
> Peter Macko
> Harvard School of Engineering and Applied Sciences
> 33 Oxford St.
> Cambridge, MA 02138
> IPython-dev mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-dev