[Numpy-discussion] numpy.random and multiprocessing

Pauli Virtanen pav@iki...
Thu Dec 11 11:03:41 CST 2008


Thu, 11 Dec 2008 17:55:58 +0100, Sturla Molden wrote:
[clip]
> Sure, a pool is fine. I was just speculating that one of the four
> processes in your pool was idle all the time; i.e. that one of the other
> three got to do the task twice. Therefore you only got three identical
> results and not four. It depends on how the OS schedules the processes,
> the number of logical CPUs, etc. You have no control over that. But if
> you had used N instances of multiprocessing.Pool instead, all N results
> should have been identical (if the 'random' generator is completely
> deterministic) - because each process would do the task once.
> 
> I.e. you only got three indentical results due to a race condition in
> the task queue.

Exactly, change task_helper.py to

----
import numpy as np

def task(x):
    import os
    print "Hi, I'm", os.getpid()
    return np.random.random(x)
----

and note the output

----
Hi, I'm 16197
Hi, I'm 16198
Hi, I'm 16199
Hi, I'm 16199
[ 0.58175647  0.16293922  0.30488182  0.67367263]
[ 0.58175647  0.16293922  0.30488182  0.67367263]
[ 0.58175647  0.16293922  0.30488182  0.67367263]
[ 0.59574921  0.61554857  0.06155764  0.75352295]
----

-- 
Pauli Virtanen



More information about the Numpy-discussion mailing list