[Numpy-discussion] copying ctypes arrays to numarray?

[Todd Miller]

I agree that numarray's .info() would be better as a string,  but I've
been leery of changing it.  Before we go to far with this,  try:

repr(src._data)

to get a string like:

"<memory at 00815950 with size:256 held by object 00815930 aliasing object 00000000>"

Note:  this technique depends on the buffer object used as the basis for
the array.  Here the repr of numarray's memory object is giving useful
information.  Other buffers, e.g. a string, might not.

It looks to me like you forgot to import Numeric so I don't think
there's any numarray:Numeric comparison going on... either that or I
just don't understand what you're to compare.

To be safe, check the .is_c_array() method.

Regards,
Todd

On Thu, 2004-12-16 at 13:33 -0800, rays wrote:
> I'm re-posting this here to see if anyone can shed light on
> numarray.info() behavior, and why assignment from a (ctypes.c_long *
> 2000)() to a numarray is so much slower than a memmove().
> 
> I was surprised that assignment wasn't faster and that numarray
> assignment was consistently ~.5% faster than Numeric.
> The N vs. n memmove() flip-flopped for fastest, with array.array
> always slower.
> 
> After some discussion, I made a better direct comparison of ctypes and
> memmove, map etc., using all the methods suggested so far. I think
> it's fairly valid.(?) Run on a 2.4GHz WinXP, Py2.3, a number of times.
> 
> Parsing numarray.info() was a royal pain, it writes directly to
> stdout!
> I.e.:
> >>> inf = src.info()
> class: <class 'numarray.numarraycore.NumArray'>
> shape: (32,)
> strides: (8,)
> byteoffset: 0
> bytestride: 8
> itemsize: 8
> aligned: 1
> contiguous: 1
> data: <memory at 00815950 with size:256 held by object 00815930
> aliasing object
> 00000000>
> byteorder: little
> byteswap: 0
> type: Float64
> >>> inf
> >>> type(inf)
> <type 'NoneType'>
> 
> 
> Since the ctype does support slicing, I was considering leaving the
> data in the device driver buffer (~1MB circular) and poking into it,
> but memmove is so much faster than slicing the ctype, I'm doing
> memmove()s to numarray.
> I presume that I should check for numarray.iscontiguous( ) or
> is_c_array( )  first to be safe...
> 
> Results and code below.
> 
> Thanks to all for the help,
> Ray
> 
> 
> 
> >python test.py
> Array address 11443208
> n address 17039424
> N address 21102656
> for loop  2027.0 us ea
> map       1781.0 us ea
> slice     1704.0 us ea
> assign  N 1244.0 us ea
> assign  n 1242.0 us ea
> memmove   4.3831 us ea
> memmove N 3.4773 us ea
> memmove n 3.4803 us ea
> 
> _____________________________________________________
> 
> ## test.py
> import array
> import numarray
> import ctypes
> import time
> import string
> import StringIO
> import sys
> 
> buf = (ctypes.c_long * 2000)()
> Array = array.array("l", [0]*10000)
> n = numarray.zeros((1000000), numarray.Int32)
> N = numarray.zeros((1000000), numarray.Int32)
> 
> #!!!!!!!!!!!!!!! arrrrgggg!  !!!!!!!!!!!!!!!!
> # n.info() writes directly to stdout!
> stdout = sys.stdout
> fileo = StringIO.StringIO()
> sys.stdout = fileo
> n.info()
> ninfo = fileo.getvalue( )
> fileo.close()
> sys.stdout = stdout
> 
> stdout = sys.stdout
> fileo = StringIO.StringIO()
> sys.stdout = fileo
> N.info()
> Ninfo = fileo.getvalue( )
> fileo.close()
> sys.stdout = stdout
> #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> 
> print 'Array address', Array.buffer_info()[0]
> 
> ninfo = string.split(ninfo)
> nAddress = int(ninfo[20], 16)
> print 'n address', nAddress
> Ninfo = string.split(Ninfo)
> NAddress = int(Ninfo[20], 16)
> print 'N address', NAddress
> 
> t0 = time.clock()
> for loop in range(1000):
>     for i in range(2000):
>         n[loop+i] = buf[i]
> print 'for loop ', round((time.clock()-t0)*1000), 'us ea'
> 
> t0 = time.clock()
> for loop in range(1000):
>     n[loop:loop+2000] = map(None, buf)
> print 'map      ', round((time.clock()-t0)*1000), 'us ea'
> 
> t0 = time.clock()
> for loop in range(1000):
>     n[loop:loop+2000] = buf[0:2000]
> print 'slice    ', round((time.clock()-t0)*1000), 'us ea'
> 
> t0 = time.clock()
> for loop in range(10000):
>     N[loop:loop+2000] = buf
> print 'assign  N', round((time.clock()-t0)*100), 'us ea'
> 
> t0 = time.clock()
> for loop in range(10000):
>     n[loop:loop+2000] = buf
> print 'assign  n', round((time.clock()-t0)*100), 'us ea'
> 
> t0 = time.clock()
> for loop in range(10000):
>     ctypes.memmove(10+Array.buffer_info()[0],
>                      buf,
>                      2000)
> print 'memmove  ', round((time.clock()-t0)*1, 4), 'us ea'
> 
> t0 = time.clock()
> for loop in range(10000):
>     ctypes.memmove(10+NAddress,
>                      buf,
>                      2000)
> print 'memmove N', round((time.clock()-t0)*1, 4), 'us ea'
> 
> t0 = time.clock()
> for loop in range(10000):
>     ctypes.memmove(10+nAddress,
>                      buf,
>                      2000)
> print 'memmove n', round((time.clock()-t0)*1, 4), 'us ea'
> 
> 
> 
> 
> 
> 
>