memory position of numpy arrays
david at ar.media.kyoto-u.ac.jp
Tue Oct 24 01:29:21 CDT 2006
Lars Friedrich wrote:
> thanks for your fast answer.
> Am Montag, den 23.10.2006, 00:45 -0700 schrieb Andrew Straw:
>> It sounds like your hardware drivers may be buggy -- you should only get
>> segfaults, not (the Windows equivalent of) kernel panics, when your
>> userspace code accesses wrong memory.
> When I started to write this thing, I had a lot of bluescreens. I think
> they occured when my Python program and the hardware driver were
> accessing the same memory location at the same time. I think the
> hardware uses some DMA-technique.
I don't know anything about your device, but a driver directly accessing
a memory buffer from a userland program sounds like a bug to me. I am
far from being knowledgeable about os programming, but kernel and user
space are two different adress spaces, so I don't see how a user-land
program could access directly memory from kernel-space (the buffer from
your hardware). There has to be copy somewhere, or fancier methods of
sharing data; otherwise, this is really unsafe (using data from userland
in kernel land is sure to cause problems....).
>> But if you have buggy hardware drivers, I suppose it's possible that
>> locking the memory will help. This wouldn't be the domain of numpy,
>> however. In Linux, this is acheived with a system call such as mlock()
>> or mlockall(). You'll have to figure out the appropriate call in
>> Windows. Thinking about it, if your hardware drivers require the memory
>> to be locked, they should do it themselves.
> I am not really sure about the term "locking". Does that mean that this
> part is not paged, or that this part is not accessed by two entities at
> the same time? Or both?
> Would I do these mlock()-calls in my C-Dll? If Yes, what would happen,
> if I tried to access the numpy-array from python, during the time it is
In this context, this means preventing the memory manager from putting
the corresponding pages to the hard-drive. This is actually part of
POSIX (I cannot remember which one), not linux-specific. Windows has a
similar API, but I remember having seen somewhere that it is not
reliable (ie it can swap pages out); I cannot find the corresponding API
> It is a camera with its own PCI-frame-grabber card. I am using it in
> "continuous-acquisition-mode" so I suppose, the driver is ready for
> *long* uses... Anyway, if the bluescreens continue I will have to switch
> to the "single-frame-acquisition-mode" and prepare a single buffer for
> every frame to grab.
> Of course, there would be a different way: I could allocate the buffer
> in C, in the dll. After data retrievel using the hardware API I could
> then copy the data to some numpy-array, still using C. But this would
> make my C-coded dll longer, thus harder to maintain.
I don't understand this either: how allocating in C changes the problem ?
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
More information about the Numpy-discussion