[Scipy-tickets] [SciPy] #1370: Running scipy.weave programs on cluster with shared filesystem

SciPy Trac scipy-tickets@scipy....
Sun Jan 23 10:52:34 CST 2011


#1370: Running scipy.weave programs on cluster with shared filesystem
-----------------------+----------------------------------------------------
 Reporter:  temporaer  |       Owner:  somebody
     Type:  defect     |      Status:  new     
 Priority:  normal     |   Milestone:  0.9.0   
Component:  Other      |     Version:  0.7.0   
 Keywords:             |  
-----------------------+----------------------------------------------------
 my application makes use of `scipy.weave` (0.4.9) to run c-code on `numpy`
 arrays. The same function is called with different data on multiple
 computers of our cluster using the parallel execution mechanisms coming
 with `ipython`.

 As far as I can see, weave stores its compiled libraries in
 `~/.python26_compiled`. As all computers share the file system via `NFS`,
 this will actually be the same folder, and the *same library file name*,
 resulting in write conflicts (and /weird/ error messages)

 I put a symbolic link from `/tmp` to `~/.python26_compiled` and that
 seemed to fix the problem.

 Should there be a way to specify and create directories elsewhere and/or
 include the hostname in the hash-generation? Or am I missing something
 important?

-- 
Ticket URL: <http://projects.scipy.org/scipy/ticket/1370>
SciPy <http://www.scipy.org>
SciPy is open-source software for mathematics, science, and engineering.


More information about the Scipy-tickets mailing list