[IPython-User] engines dying due to excessive load

[Fernando Perez]

On Fri, Jun 29, 2012 at 9:22 AM, Robert Nishihara
<robertnishihara@gmail.com> wrote:
> Ok, so numpy uses Intel's Math Kernel Library, which tries to automatically
> parallelize things, which, on a cluster, can cause problems for the
> scheduler.
>
> Setting MLK_NUM_THREADS=1 on the engines appears to have completely fixed
> the problem.

This is the classic parallelization battle between libraries at
multiple levels, which to the best of my understanding, can only be
solved correctly if all codes delegate thread creation to a central
point such as apple's Grand Central Dispatch
(http://arstechnica.com/apple/2009/08/mac-os-x-10-6/12).

Cheers,

f