[Numpy-discussion] bus error in embedded numpy

Robin robince@gmail....
Fri Nov 13 14:10:54 CST 2009


On Fri, Nov 13, 2009 at 7:48 PM, Sturla Molden <sturla@molden.no> wrote:
> Robin skrev:
>> I had assumed when matlab unloads the mex function it would also
>> unload python - but it looks like other dynamic libs pulled in from
>> the mex function (in this case python and in turn numpy) aren't
>> unloaded...
>>
> Matlab MEX functions are DLLs, Python interpreter is a DLL, NumPy .pyd
> files are DLLs... If you cannot unload Python or NumPy, you cannot
> unload MEX functions either. The same OS constraints apply.

Ah, I hadn't realised it was an OS constraint - I thought it was
possible to unload dlls - and that was why matlab provides the clear
function. mex automatically clears a function when you rebuild it - I
thought that was how you can rebuild and reload mex functions without
restarting matlab. I thought it was just a quirk of matlab that there
was no way to unload other libraries pulled in through the mex
function. I must admit to being a bit out of my depth with this stuff
though so I stand corrected.

> If you are using Windows, I would recommend that you expose your NumPy
> code as a COM object using pywin32, and make a Matlab wrapper (ordinary
> .m file) that calls the COM object. If you are not using Windows, you
> could create an XMLRPC server in Python and call that using Matlab's
> built-in Java VM.
>
> Or you can spawn a separate Python process from Matlab, and communicate
> using pipes or a tcp/ip socket (Matlab's Java or MEX).
>
> There are many solutions that are easier than embedding Python in Matlab.

I really want a cross platform solution so that rules out COM. I
thought about using web services or something which I guess would be
the easiest way to do communication through a tcp socket (least work
since I could use existing web services libraries on both sides). But
actually I have found it pretty easy to embed Python so far... with
about 5 lines of code I'm able run simple strings and with a small
cython module I get stdout to the console. I haven't tried passing
data back and forth yet but from Pauli's pythoncall it doesn't look
like that is too difficult. I was sort of hoping that eventually I
could create a numpy array from a matlab data pointer - at least for
read only input - so that I could use python code to work on large
amounts of data without so much memory overhead (returning smaller
results by copying).

Do you think I'm likely to run into more problems? I got the feeling
from asking questions on IRC that embedding Python is kind of
discouraged but I thought in this case it seemed the neatest way.

> Be careful when you are using file handles. You have to be sure that
> Matlab, Python and NumPy are all linked against the same CRT.

I was aware of this - I thought I would be OK on the mac - at least
Python and Numpy and my mex function are built with apple gcc although
I'm not sure about Matlab. I guess Windows will be more difficult...
But in any case I don't plan to pass any file handles around.

Cheers

Robin


More information about the NumPy-Discussion mailing list