[IPython-dev] Project idea: Automatic lab notebook for iPython
Fri Apr 19 07:21:46 CDT 2013
also do look at emerging provenance standard:
On Fri, Apr 19, 2013 at 5:28 AM, Maximilian Albert <
> Hi Peter,
> I wasn't aware of the Core Provenance Library but it looks like a very
> interesting project. I'll definitely check it out.
> On a related note, in case people haven't seen it I just wanted to throw
> out a link to the Sumatra  project, which has a similar goal. However,
> as far as I could gather from its website, the Core Provenance Library
> seems to take a very different approach (by trying to be as unobtrusive for
> the workflow as possible, IIUC?). So I guess the link is not that useful
> for you, Peter, but I thought it might be of general interest. If anybody
> knows any other provenance tracking tools, I'd be interested to hear about
> Best regards,
>  http://pythonhosted.org/Sumatra/
> 2013/4/17 Peter Macko <email@example.com>
>> Hi iPython developers,
>> Here is a new project idea: automatic lab notebook for iPython and
>> iPython Notebook, which would keep track of how each of your output
>> files was produced, linking this "history" (or a "lineage") of an object
>> across different iPython sessions and different iPython notebooks, and
>> storing it persistently. This is frequently referred to in the Computer
>> Science literature as "provenance."
>> It will enable you to ask questions like "what did I do to produce this
>> plot?" - and for example, it will tell you that you downloaded the input
>> data set on Monday from such and such website, you ran all these
>> commands to process the data on Tuesday, and then produced this plot on
>> Thursday from a different iPython session. Note that this goes beyond
>> (and is complementary in purpose to) iPython Notebook, since the history
>> of a file is tracked across different sessions and Notebooks, and when
>> you ask a question, you will get only the relevant information,
>> suppressing any additional things that you did that are unrelated to the
>> file in which you are interested.
>> We are in touch with computational scientists all the way from
>> bioinformatics to physics that are very interested in this feature! We
>> met their needs partially by developing a cross-platform, multi-lingual
>> library (https://code.google.com/p/core-provenance-library/) that they
>> can use to annotate their Python (and non-Python) scripts in order to
>> track the lineage of their objects.
>> Our vision is that this will be all done fully automatically, without
>> requiring the users to manually annotate their scripts. But
>> unfortunately neither of us who are involved in this project has the
>> resources or the knowledge of the iPython code-base to tackle this
>> challenge. We need your help to make this happen! We have some ideas
>> about how we might go about this, but we need someone who knows more
>> about iPython to talk them over and to spearhead the actual development.
>> Please let us know if you can help!
>> Thank you,
>> Peter Macko
>> Harvard School of Engineering and Applied Sciences
>> 33 Oxford St.
>> Cambridge, MA 02138
>> IPython-dev mailing list
> IPython-dev mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-dev