[SciPy-User] multiprocessing module

Ted To forums@theo...
Mon Nov 1 17:42:05 CDT 2010


On Mon, Nov 1, 2010 at 5:36 PM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
>> I'm trying to get multiprocess to do a bunch of independent
>> calculations and save the results in a file and I'm probably going
>> about it the wrong way.  I have a function defined "computeEq" that
>> does the calculation and writes the result to a file (outfile) and I
>> call it using:
>>
>> po = Pool()
>> po.map_async(computeEq, product(rules,repeat=N))
>> po.close()
>> po.join()
>> outfile.close()
>>
>> This seems to work for the most part but I seem to lose the last few
>> calculations.  Indeed, one of my writes is truncated before the write
>> is complete.
>
> Are you taking proper precautions so that multiple workers aren't
> trying to write to the file at the same time?
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>

I'm a bit of a noob as far as multiprocessing goes so no, I'm not.
How does one do that?

Thanks,
Ted


More information about the SciPy-User mailing list