[IPython-dev] DAG Dependencies
Thu Oct 28 14:34:45 CDT 2010
On Thu, Oct 28, 2010 at 12:00 PM, Fernando Perez <email@example.com> wrote:
> On Thu, Oct 28, 2010 at 11:55 AM, MinRK <firstname.lastname@example.org> wrote:
>> My interactive session starts running into memory issues with 100M ids, so
>> that's something to consider.
> Well, let's just hope that anyone with 100M ids in real life (not a
> synthetic benchmark) has the common sense of working on something with
> 100G of RAM or so :)
But, with the performance we are getting, some people might want to
simply run this 24/7 (instead of using a batch system). Then, over a
long period of time, having that many tasks is not too crazy. We
definitely need to think about this aspect of things.
> Thanks a lot for looking into this, Min. Fantastic example to have,
> and one that is very relevant to many use cases.
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
More information about the IPython-dev