[IPython-dev] Project idea: Automatic lab notebook for iPython
Fri Apr 19 04:28:49 CDT 2013
I wasn't aware of the Core Provenance Library but it looks like a very
interesting project. I'll definitely check it out.
On a related note, in case people haven't seen it I just wanted to throw
out a link to the Sumatra  project, which has a similar goal. However,
as far as I could gather from its website, the Core Provenance Library
seems to take a very different approach (by trying to be as unobtrusive for
the workflow as possible, IIUC?). So I guess the link is not that useful
for you, Peter, but I thought it might be of general interest. If anybody
knows any other provenance tracking tools, I'd be interested to hear about
2013/4/17 Peter Macko <email@example.com>
> Hi iPython developers,
> Here is a new project idea: automatic lab notebook for iPython and
> iPython Notebook, which would keep track of how each of your output
> files was produced, linking this "history" (or a "lineage") of an object
> across different iPython sessions and different iPython notebooks, and
> storing it persistently. This is frequently referred to in the Computer
> Science literature as "provenance."
> It will enable you to ask questions like "what did I do to produce this
> plot?" - and for example, it will tell you that you downloaded the input
> data set on Monday from such and such website, you ran all these
> commands to process the data on Tuesday, and then produced this plot on
> Thursday from a different iPython session. Note that this goes beyond
> (and is complementary in purpose to) iPython Notebook, since the history
> of a file is tracked across different sessions and Notebooks, and when
> you ask a question, you will get only the relevant information,
> suppressing any additional things that you did that are unrelated to the
> file in which you are interested.
> We are in touch with computational scientists all the way from
> bioinformatics to physics that are very interested in this feature! We
> met their needs partially by developing a cross-platform, multi-lingual
> library (https://code.google.com/p/core-provenance-library/) that they
> can use to annotate their Python (and non-Python) scripts in order to
> track the lineage of their objects.
> Our vision is that this will be all done fully automatically, without
> requiring the users to manually annotate their scripts. But
> unfortunately neither of us who are involved in this project has the
> resources or the knowledge of the iPython code-base to tackle this
> challenge. We need your help to make this happen! We have some ideas
> about how we might go about this, but we need someone who knows more
> about iPython to talk them over and to spearhead the actual development.
> Please let us know if you can help!
> Thank you,
> Peter Macko
> Harvard School of Engineering and Applied Sciences
> 33 Oxford St.
> Cambridge, MA 02138
> IPython-dev mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-dev