[Numpy-discussion] Using multiprocessing (shared memory) with numpy array multiplication

Brandt Belson bbelson@princeton....
Fri Jun 10 10:01:48 CDT 2011


Unfortunately I can't flatten the arrays. I'm writing a library where the
user supplies an inner product function for two generic objects, and almost
always the inner product function does large array multiplications at some
point. The library doesn't get to know about the underlying arrays.
Thanks,
Brandt



> Message: 2
> Date: Fri, 10 Jun 2011 09:23:10 -0400
> From: Olivier Delalleau <shish@keba.be>
> Subject: Re: [Numpy-discussion] Using multiprocessing (shared memory)
>        with numpy array multiplication
> To: Discussion of Numerical Python <numpy-discussion@scipy.org>
> Message-ID: <BANLkTikjppC90yE56T1mr+byAxXAw32YJA@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> It may not work for you depending on your specific problem constraints, but
> if you could flatten the arrays, then it would be a dot, and you could
> maybe
> compute multiple such dot products by storing those flattened arrays into a
> matrix.
>
> -=- Olivier
>
> 2011/6/10 Brandt Belson <bbelson@princeton.edu>
>
> > Hi,
> > Thanks for getting back to me.
> > I'm doing element wise multiplication, basically innerProduct =
> > numpy.sum(array1*array2) where array1 and array2 are, in general,
> > multidimensional. I need to do many of these operations, and I'd like to
> > split up the tasks between the different cores. I'm not using numpy.dot,
> if
> > I'm not mistaken I don't think that would do what I need.
> > Thanks again,
> > Brandt
> >
> >
> > Message: 1
> >> Date: Thu, 09 Jun 2011 13:11:40 -0700
> >> From: Christopher Barker <Chris.Barker@noaa.gov>
> >> Subject: Re: [Numpy-discussion] Using multiprocessing (shared memory)
> >>        with numpy array multiplication
> >> To: Discussion of Numerical Python <numpy-discussion@scipy.org>
> >> Message-ID: <4DF128FC.8000807@noaa.gov>
> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >>
> >> Not much time, here, but since you got no replies earlier:
> >>
> >>
> >> >      > I'm parallelizing some code I've written using the built in
> >> >     multiprocessing
> >> >      > module. In my application, I need to multiply many large arrays
> >> >     together
> >>
> >> is the matrix multiplication, or element-wise? If matrix, then numpy
> >> should be using LAPACK, which, depending on how its built, could be
> >> using all your cores already. This is heavily dependent on your your
> >> numpy (really the LAPACK it uses0 is built.
> >>
> >> >      > and
> >> >      > sum the resulting product arrays (inner products).
> >>
> >> are you using numpy.dot() for that? If so, then the above applies to
> >> that as well.
> >>
> >> I know I could look at your code to answer these questions, but I
> >> thought this might help.
> >>
> >> -Chris
> >>
> >>
> >>
> >>
> >>
> >> --
> >> Christopher Barker, Ph.D.
> >> Oceanographer
> >>
> >> Emergency Response Division
> >> NOAA/NOS/OR&R            (206) 526-6959   voice
> >> 7600 Sand Point Way NE   (206) 526-6329   fax
> >> Seattle, WA  98115       (206) 526-6317   main reception
> >>
> >> Chris.Barker@noaa.gov
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20110610/d7689cba/attachment.html 


More information about the NumPy-Discussion mailing list