[SciPy-dev] Implementation of a parallel cKDTree

[Sturla Molden]

I have fiddled a bit with scipy.spatial.cKDTree for better performance 
on multicore CPUs. I have used threading.Thread instead of OpenMP, so no 
special compilation or compiler is required. The number of threads 
defaults to the number of processors if it can be determined. The 
performance is not much different from what I get with OpenMP. It is 
faster than using cKDTree with multiprocessing and shared memory.

Memory handling is also improved. There are checks for NULL pointers 
returned by malloc or realloc. setjmp/longjmp is used for error handling 
if malloc or realloc fail. A memory pool is used to make sure all 
complex data structures are cleaned up properly.

I have assumed that crt functions malloc, realloc and free are thread 
safe. This is usually the case. If they are not, they must be wrapped 
with calls to PyGILState_Ensure and PyGILState_Release. I have not done 
this as it could impair scalability.


Regards,
Sturla Molden
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ckdtree_mt.pyx
Type: /
Size: 29331 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/scipy-dev/attachments/20090227/e4131bd2/attachment-0001.bin