[Numpy-discussion] numpy : your experiences?

[Steve Lianoglou]

Hi Rahul,

> a) Can you guys tell me briefly about the kind of problems you are
> tackling with numpy and scipy?

I'm a grad student "doing" computational biology.  I primarily use the  
NumPy/SciPy/matplotlib triumvirate as a post processing tool to  
analyze what the heck happened after we run some learning algorithms  
we develop (or canned ones, like libsvm (for example)) to look for  
some sense in the results.

I've been working w/ analyzing interaction networks/graphs, so I also  
use NetworkX[1] quite a bit as well (it's also a nice package w/  
responsive authors).

Many of the folks (in my lab, and collaborators) like to use MATLAB,  
so I've found scipy's io.loadmat invaluable for making this a bit more  
seamless.

So, in general, for me (so far) numpy/scipy are generally used to  
integrate various datasets together and see if things "look  
kosher" (before runs and after runs).

> b) Have you ever felt that numpy/scipy was slow and had to switch to
> C/C++/Fortran?

Yes, for things like boosting, svm, graph mining, etc ... but that's  
no real surprise since their iterative and need to run on large  
datasets.

You should also note that there are python interfaces to these things  
out there as well, but I (thus far) haven't taken much of advantage of  
those and usually pipe out data into the expected text input formats  
and pull them back in when the algo is done.

> c) Do you use any form of parallel processing? Multicores? SMPs?
> Clusters? If yes how did u utilize them?

I'd really like to (not just for Python), but I haven't.

-steve

[1] NetworkX: https://networkx.lanl.gov/wiki