[Numpy-discussion] Windows, blas, atlas and dlls

Dag Sverre Seljebotn d.s.seljebotn@astro.uio...
Mon Feb 18 11:20:42 CST 2013


On 02/18/2013 05:29 PM, rif wrote:
> But I'd hope that the overhead for going through the wrappers is
> constant, rather than dependent on the size, so that for large matrices
> you'd get essentially equivalent performance?

That is correct.

Ah, so then the quality of the BLAS matters much less in this situation.

But if you have a code that is used with either many small or fewer 
large matrices, then a compiled loop over a good BLAS is a good 
compromise without splitting up the code paths.

DS

>
>
> On Mon, Feb 18, 2013 at 8:28 AM, Dag Sverre Seljebotn
> <d.s.seljebotn@astro.uio.no <mailto:d.s.seljebotn@astro.uio.no>> wrote:
>
>     On 02/18/2013 05:26 PM, rif wrote:
>      > I have no answer to the question, but I was curious as to why
>     directly
>      > calling the cblas would be 10x-20x slower in the first place.  That
>      > seems surprising, although I'm just learning about python numerics.
>
>     The statement was that directly (on the Cython level) calling cblas is
>     10x-20x slower than going through the (slow) SciPy wrapper routines.
>     That makes a lot of sense if the matrices are smalle nough.
>
>     Dag Sverre
>
>      >
>      >
>      > On Mon, Feb 18, 2013 at 7:38 AM, Sergio Callegari
>      > <sergio.callegari@gmail.com <mailto:sergio.callegari@gmail.com>
>     <mailto:sergio.callegari@gmail.com
>     <mailto:sergio.callegari@gmail.com>>> wrote:
>      >
>      >     Hi,
>      >
>      >     I have a project that includes a cython script which in turn does
>      >     some direct
>      >     access to a couple of cblas functions. This is necessary,
>     since some
>      >     matrix
>      >     multiplications need to be done inside a tight loop that gets
>     called
>      >     thousands
>      >     of times. Speedup wrt calling scipy.linalg.blas.cblas routines is
>      >     10x to 20x.
>      >
>      >     Now, all this is very nice on linux where the setup script can
>      >     assure that the
>      >     cython code gets linked with the atlas dynamic library, which
>     is the
>      >     same
>      >     library that numpy and scipy link to on this platform.
>      >
>      >     However, I now have trouble in providing easy ways to use my
>     project in
>      >     windows. All the free windows distros for scientific python
>     that I have
>      >     looked at (python(x,y) and winpython) seem to repackage the
>     windows
>      >     version of
>      >     numpy/scipy as it is built in the numpy/scipy development sites.
>      >     These appear
>      >     to statically link atlas inside some pyd files.  So I get no
>     atlas
>      >     to link
>      >     against, and I have to ship an additional pre-built atlas with my
>      >     project.
>      >
>      >     All this seems somehow inconvenient.
>      >
>      >     In the end, when my code runs, due to static linking I get 3
>      >     replicas of 2
>      >     slightly different atlas libs in memory. One coming with
>      >     _dotblas.pyd in numpy,
>      >     another one with cblas.pyd or fblas.pyd in scipy. And the
>     last one
>      >     as the one
>      >     shipped in my code.
>      >
>      >     Would it be possible to have a win distro of scipy which
>     provides some
>      >     pre built atlas dlls, and to have numpy and scipy dynamically
>     link
>      >     to them?
>      >     This would save memory and also provide a decent blas to link
>     to for
>      >     things
>      >     done in cython. But I believe there must be some problem
>     since the
>      >     scipy site
>      >     says
>      >
>      >     "IMPORTANT: NumPy and SciPy in Windows can currently only
>     make use
>      >     of CBLAS and
>      >     LAPACK as static libraries - DLLs are not supported."
>      >
>      >     Can someone please explain why or link to an explanation?
>      >
>      >     Unfortunately, not having a good, prebuilt and cheap blas
>      >     implementation in
>      >     windows is really striking me as a severe limitation, since you
>      >     loose the
>      >     ability to prototype in python/scipy and then move to C or Cython
>      >     the major
>      >     bottlenecks to achieve speed.
>      >
>      >     Many thanks in advance!
>      >
>      >     _______________________________________________
>      >     NumPy-Discussion mailing list
>      > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>     <mailto:NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>>
>      > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>      >
>      >
>      >
>      >
>      > _______________________________________________
>      > NumPy-Discussion mailing list
>      > NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>      > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>      >
>
>     _______________________________________________
>     NumPy-Discussion mailing list
>     NumPy-Discussion@scipy.org <mailto:NumPy-Discussion@scipy.org>
>     http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>



More information about the NumPy-Discussion mailing list