[Numpy-discussion] numpy : your experiences?

[Konrad Hinsen]

On 17.11.2007, at 03:50, Rahul Garg wrote:

> It would be awesome if you guys could respond to some of the following
> questions :
> a) Can you guys tell me briefly about the kind of problems you are
> tackling with numpy and scipy?

For me, NumPy is an important building block in a set of  
computational Python libraries that form the basis of my daily work  
in molecular simulations. The main building block is my Molecular  
Modelling Toolkit (http://dirac.cnrs-orleans.fr/MMTK/), but NumPy  
provides the basic data structure (arrays) for much of what MMTK  
does, and even more so for interfacing with the rest of the world. I  
don't use SciPy at all.

> b) Have you ever felt that numpy/scipy was slow and had to switch to
> C/C++/Fortran?

There is some C code (and an increasing amount of Pyrex code) in my  
Python environment, and it is essential for good performance.  
However, in terms of code quantity, it is negligible.

> c) Do you use any form of parallel processing? Multicores? SMPs?
> Clusters? If yes how did u utilize them?

All of them. I use threading (multicores and SMPs) in MMTK, and  
coarse-grained parallelization as implemented in ScientificPython for  
analyzing large data sets.

ScientificPython has two parallel computing modules. The easiest to  
use implements a master-slave model in which a master process  
delegates computational tasks to an arbitrary (and possibly varying)  
number of slave processes:

http://dirac.cnrs-orleans.fr/hg/ScientificPython/main/file/ 
73cc270217fc/Examples/master_slave_demo.py

The other parallelization package is based on the BSP model of  
parallel computing:

http://dirac.cnrs-orleans.fr/hg/ScientificPython/main/file/ 
73cc270217fc/Examples/BSP/
http://dirac.cnrs-orleans.fr/ScientificPython/ScientificPythonManual/

It probably has a steeper learning curve, but it is suitable for more  
complex parallel programs because it permits the construction of  
parallel libraries.

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen@cnrs-orleans.fr
---------------------------------------------------------------------