[Numpy-discussion] Objected-oriented SIMD API for Numpy
Thu Oct 22 12:42:42 CDT 2009
Robert Kern skrev:
> I would be delighted to see a reference to one that refers to a high
> level language's API as SIMD. Please point one out to me. It's
> certainly not any of the ones I have available to me.
Numerical Receipes in Fortran 90, page 964 and 985-986, describes the
syntax of Fortran 90 and 95 as SIMD.
Peter Pacheco's book on MPI describes the difference between von Neumann
machines and vector machines as analogous to the difference between
Fortran77 and Fortran 90 (with an example from Fortran90 array slicing).
He is ambigous as to whether vector machines really are SIMD, or more
related to pipelined von Neumann machines.
Grama et al. "Introduction to Parallel Computing" describes SIMD as an
"architecture", but it is more or less clear that the mean hardware.
They do say the Fortran 90 "where statement" is a primitive used to
support selective execution on SIMD processors, as conditional execution
(if statements) are detrimental to performance.
So at least we here have three books claiming that Fortran is a language
with special primities for SIMD processors.
> That's a fair point, but unrelated to whether or not numpy can be
> labeled SIMD. These all refer to hardware.
Actually I don't think the distinction is that important as we are
taking about Turing machines. Also, a lot of what we call "hardware" is
actually implemented as software on the chip: The most extreme example
would be Transmeta, which completely software emulated x86 processors.
The vague distinction between hardware and software is why we get
patents on software in Europe, although pure software patents are
prohibited. One can always argue that the program and the computer
together constitutes a physical device; and circumventing patents by
moving hardware into software should not be allowed. The distinction
between hardware and software is not as clear as programmers tend to
Another thing is that performance issues for vector machines and "vector
languages" (Fortran 90, Matlab, NumPy) are similar. Precisely the same
situations that makes NumPy and Matlab code slow are detrimental on
SIMD/vector hardware. That would for example be long for loops with
conditional if statements. On the other hand, vectorized operations over
arrays, possibly using where/find masks, are fast. So although NumPy is
not executed on a vector machine like the Cray C90, it certainly behaves
like one performance wise.
I'd say that a MIMD machine running NumPy is a Turing machine emulating
a SIMD/vector machine.
And now I am done with this stupid discussion...
More information about the NumPy-Discussion