[SciPy-User] 64-bit matrix indices
Tue Nov 16 10:09:02 CST 2010
I am trying to use SciPy for one of my HPC projects. A problem I am
currently facing is that 32-bit indices are too small for the matrix sizes
we require. Is there any way to use 64-bit ints for the indices?
I don't mind if that requires extending/modifying SciPy (although it might
result in severe disturbance in the world supply of coffee since I'd have
to do it in about a week) -- the other alternative I have is coding the
whole thing in C++ which is exactly what I'm trying to avoid. However, I
am somewhat confused by the datatype definitions. From the little amount
of source diving I've done, it would seem that the datatype for the
indices is numpy.intc which ends up int32 (I'm not sure why it does -- I
was thinking it should be int64 on a 64-bit machine). Is this critical? If
so, why? Is it possible to set this when compiling NumPy or SciPy?
If not, what would be a better approach? I was thinking about hardcoding
dtype=int64 for the array of indices scipy uses (and maybe using a custom
sparse class for it so as not to require our collaborators to drag a
custom-built and modified version of scipy along with them), but I am not
sure about the implications this would have towards the rest of the SciPy
build. I only need a few sparse operations and .mat file reading from it.
If anyone is interested on the background story: the matrices themselves
aren't too big *at first*, but due to the peculiar structure they have,
the fill-in is mind-blowing. UMFPACK complaints that it doesn't have
enough memory for them; it does (our cluster's nodes have 24 GB of
memory), but once the number of indices blows past the 32-bit limit, it
hits the ceiling. Using a different solver is still not an option
(SuperLU, which as far as I understand is the current default in SciPy,
doesn't have the peculiar way of retaining complex numbers UMFPACK has; it
would allow me to go twice as far, but I need to go about 3-4 times
farther than that). tl ; dr 32-bit indices are enough to retain any system
matrix, but not enough to cope with the fill-in of the L and U factors,
regardless of how awesomely I preorder them.
More information about the SciPy-User