memory position of numpy arrays
strawman at astraw.com
Mon Oct 23 02:45:19 CDT 2006
It sounds like your hardware drivers may be buggy -- you should only get
segfaults, not (the Windows equivalent of) kernel panics, when your
userspace code accesses wrong memory.
But if you have buggy hardware drivers, I suppose it's possible that
locking the memory will help. This wouldn't be the domain of numpy,
however. In Linux, this is acheived with a system call such as mlock()
or mlockall(). You'll have to figure out the appropriate call in
Windows. Thinking about it, if your hardware drivers require the memory
to be locked, they should do it themselves. However, I'm not convinced
this is the real issue. It seems at least equally likely that your
hardware drivers were developed with particular pattern of timing when
accessing the buffers, but now you may be attempting to hold a buffer
longer (preventing the driver writing to it) than the developer ever
tested. It shouldn't blue-screen, but it does...
I think it quite likely that you have some buggy hardware drivers. What
hardware is it?
Lars Friedrich wrote:
> Hello all,
> to interact with some hardware (data retrieval), I use the following
> scheme (Windows, Python 2.4):
> * in Python, I create a numpy array as a buffer
> * I pass this array to a self written .dll using ctypes
> * the C-code in the .dll passes the pointer to the buffer to the API of
> the hardware; then the API starts writing data to my buffer
> * from python I can use a helper function of the .dll to know which part
> of the buffer is safe to be read at the moment; so I can copy this part
> to a different numpy array and work with the data
> This works quite well, but today I got a blue screen, reporting some
> paging-problem. It occured after doing some other stuff like moving some
> windows on the screen and importing pylab, and I heard the harddisk
> working hard, so I assume that the part of memory my buffer is in, was
> paged to the harddisk. This is a problem, since the hardware driver will
> continuously try to write to this specific memory location I gave to it.
> My primary question is, how to avoid a numpy-array being paged. In the
> ideal case there would be a flag to set, that makes sure that this array
> is always at this position in physical memory.
> Of course I am also interested in other people's work on hardware
> access. Do you think the above discribed way is a good one? To me it was
> the best way because I could do as much as possible in Python and keep
> may C-coded .dll very small. Is anyone doing similar things?
> Thanks for every comment
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
More information about the Numpy-discussion