[Numpy-discussion] [Jepp-users] win64 numpy hangs during tests using embedded interpreter

[Jon Wright]

Ah, bugger, found the answer and it is bad news, from 
http://docs.python.org/c-api/init.html

"Note that the PyGILState_*() functions assume there is only one global 
interpreter (created automatically by Py_Initialize() 
<http://docs.python.org/c-api/init.html#Py_Initialize>). Python still 
supports the creation of additional interpreters (using 
Py_NewInterpreter() 
<http://docs.python.org/c-api/init.html#Py_NewInterpreter>), but mixing 
multiple interpreters and the PyGILState_*() API is unsupported."

Since jepp is using Py_NewInterpreter() to allow many interpreters then 
it means it won't play nicely with extension modules who use PyGILState, 
like numpy. I can get around it by disabling threads in numpy 
altogether, but I'd rather have them working in these days of multi-core 
CPUs. Any ideas?

Thanks,

Jon

Jon Wright wrote:
> Hi There,
>
> I am trying to use numpy with jepp (from jepp.sourceforge.net) which is 
> an embedded python interpreter on win64 (vista). With the sourceforge 
> binaries I get intermittent crashes on import which went away when I 
> compiled svn trunk with msvc9. Suggests there is something incompatible 
> about them?
>
> Jepp has been compiled with Py_OptimizeFlag = 0 in pyembed_startup and 
> also removing  PyRun_SimpleString("__builtins__.__import__ = 
> jep.jimport\n");
> from pyembed_thread_init (otherwise numpy is not found).
>
> Using the python.org win64 interpreter it seems to work fine, so I'm 
> looking for advice on how to debug the problem. Compiling with debugging 
> on (edit distutils/msvc9compiler.py as described here 
> http://mail.python.org/pipermail/python-list/2003-October/229543.html ) 
> I found it is blocking here:
>
>
>  >    umath.pyd!_error_handler(int method=2, _object * 
> errobj=0x000000004d774d88, char * errtype=0x000000004c06c280, int 
> retstatus=1, int * first=0x00000000024f8de0)  Line 69 +
>
>       ALLOW_C_API;
>      000000004C04A783  call        qword ptr [__imp_PyGILState_Ensure 
> (4C0684C8h)]
> ->   000000004C04A789  mov         dword ptr [__save__],eax
>     switch(method) {
>
> Way up the call stack it seems jepp has done this to call numpy:
>
>     PyEval_AcquireLock();
>     prevThread = PyThreadState_Swap(jepThread->tstate);
> ...
>     result = PyRun_String(str,  /* = numpy.test() */
>                           Py_single_input,
>                           jepThread->globals,
>                           jepThread->globals);
>
> So it seems there is some sort of a threading problem. Does anyone have 
> any ideas?
>
> Thanks for any tips,
>
> Jon
>
>
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge  
> This is your chance to win up to $100,000 in prizes! For a limited time, 
> vendors submitting new applications to BlackBerry App World(TM) will have
> the opportunity to enter the BlackBerry Developer Challenge. See full prize  
> details at: http://p.sf.net/sfu/Challenge
> _______________________________________________
> Jepp-users mailing list
> Jepp-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/jepp-users
>