[IPython-User] distributed arrays

Brian Granger ellisonbg@gmail....
Tue Mar 8 11:37:45 CST 2011


Samuel,

> I've managed to install ipythondistarrays with fftw-3.3.alpha1. My problem
> is that I get an error when trying to run a simple test:

I think this is an issue with setting LD_LIBRARY_PATH to point to the
dir that has the .so files.  BUT (sorry if this got lost in the
discussion) the FFTW bindings have some serious bugs that will prevent
you from actually solving problems and getting the right answer.
IIRC, there are *some* cases that do work OK, but most won't.  With
this in mind, I would almost advise looking at adding support for FFTW
to pytrillinos or petsc4py or GA.  Obviously, you could fix the bug in
ipython, but because we are not working on it, I think it would make
sense to add it to a more stable project.

Cheers,

Brian

>>>> import distarray as da
>  ---> 16 from distarray.fft import py_fftw
> ImportError: libfftw3_mpi.so.3: cannot open shared object file: No such file
> or directory
> ----
>
> This happens when the module tries to import py_fftw.so. I've checked that I
> have libfftw3_mpi.so.3 in /usr/include/lib. Am I missing something? I've
> tried loading the library directly with
> ctypes.cdll.LoadLibrary('fftw3_mpi.so.3') but that didn't work either.
>
> Cheers,
>
> Sam
>
>
> On 4 March 2011 18:05, Brian Granger <ellisonbg@gmail.com> wrote:
>>
>> Samuel,
>>
>> > First of all thanks for IPython, which I think is a fantastic software.
>> > I
>> > used to work with Matlab but I have switched to Python a couple of years
>> > ago
>> > and never looked back.
>>
>> Cool!
>>
>> > I'm a postdoc working in fluid dynamics and acoustics and have to
>> > compute
>> > Fourier transforms on very big arrays (~ 4GB). I've stumbled upon this
>> > video
>> > http://www.youtube.com/watch?v=Sa3SHmiXxbw which demos distributed
>> > arrays in
>> > Python. It shows how to distribute an array seemlessly over several
>> > processors, and how to carry out sevral operations (including the fft).
>> > I
>> > was really impressed. This is exactly what I would need.
>>
>> Great.
>>
>> > However I haven't been able to find more information on this. I've only
>> > found this old Trac repository
>> >
>> > http://ipython.scipy.org/ipython/ipython/browser/ipythondistarray/trunk/ipythondistarray?rev=2990,
>> > but I don't know how to download or clone it. Is the project still
>> > active?
>> > I'd be happy to help revive this.
>>
>> The distributed array stuff has not moved forward at all and has quite
>> a few bugs in it.  But, here is the latest code:
>>
>> https://code.launchpad.net/~ipython-dev/ipython/ipythondistarray
>>
>> I would also check out the following:
>>
>> http://code.google.com/p/petsc4py/
>> http://trilinos.sandia.gov/packages/pytrilinos/
>> http://www.emsl.pnl.gov/docs/global/index.shtml
>>
>> I am sure these have fftw support, but they are much more stable and
>> it might be easy to add FFTW support.
>>
>> Cheers,
>>
>> Brian
>>
>> > Any pointers would be much appreciated.
>> >
>> > Sam
>> >
>> >
>> > _______________________________________________
>> > IPython-User mailing list
>> > IPython-User@scipy.org
>> > http://mail.scipy.org/mailman/listinfo/ipython-user
>> >
>> >
>>
>>
>>
>> --
>> Brian E. Granger, Ph.D.
>> Assistant Professor of Physics
>> Cal Poly State University, San Luis Obispo
>> bgranger@calpoly.edu
>> ellisonbg@gmail.com
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu
ellisonbg@gmail.com


More information about the IPython-User mailing list