[Numpy-discussion] Fwd: GPU Numpy

James Bergstra bergstrj@iro.umontreal...
Wed Sep 9 10:27:30 CDT 2009

On Wed, Sep 9, 2009 at 10:41 AM, Francesc Alted <faltet@pytables.org> wrote:
>> Numexpr mainly supports functions that are meant to be used element-wise,
>> so the operation/element ratio is normally 1 (or close to 1).  In these
>> scenarios is where improved memory access is much more important than CPU
>> (or, for that matter, GPU), and is the reason why numexpr is much more
>> efficient than NumPy when evaluating complex expressions like
>> ``a*b+c*sqrt(d)``.
>> In other words, a GPU-enabled numexpr makes little sense.

There's another way of looking at this, which has been mentioned
before in the conversation, but which I think should be mentioned

The cost of transfer to and from a GPU is very high, compared with
most of the sorts of things that we do with ndarrays.  So the approach
of using libraries to speed up little pieces here and there (i.e. with
VML or ATLAS) but basically to let stock numpy take care of the rest
does not work.  In order to benefit from huge speedups on a GPU, data
need to be on the GPU already.  It is a good idea to perform
low-instruction density functions on the GPU even when the CPU could
go just as fast (or even if the CPU is faster!) just to ensure that
the data stay on the GPU.

Suppose you want to evaluate "dot(a*b+c*sqrt(d), e)".  The GPU is
great for doing dot(), but if you have to copy the result of the
elemwise expression to the GPU before you can start doing dot(), then
the performance advantage is ruined.  Except for huge matrices, you
might as well just leave the data in the system RAM and use a normal
BLAS library.

So that's why it is a good idea to use the GPU to do some functions
even when the CPU would be faster for them (in isolation).

All that said, there is a possibility that future devices (and some
laptops already?) will use an integrated memory system that might make
'copying to the GPU' a non-issue... but we're not there yet I think...


More information about the NumPy-Discussion mailing list