[SciPy-user] Multithreading cookbook entry
Thu Feb 21 13:47:45 CST 2008
--- Anne Archibald <email@example.com> wrote:
> Yep. Oops. Fixed in the v2 versions of the files.
> The wiki doesn't
> make a very good version control system. Is it worth
> those files into scipy?
If you mean put a new version of the example code up
to SciPy cookbook, then yes because bugs confuse
newbies like me. :-)
> Depends what your function does, really. If your
> function takes half a
> second but never releases the GIL, it should take
> twice as long. If
> your function releases the GIL, then it should take
> about the same
> time. But it's quite tricky to write a function that
> works hard and
> releases the GIL. A good rule of thumb is to count
> the lines of python
> that are getting executed. If there are only a few -
> say you're doing
> sum(log(exp(arange(1000000)))) - there's a good
> chance the GIL will be
> released. If you're running millions of python
> instructions, the GIL
> is held all that time, and you won't get a speedup.
Hmmm... gotta think about that.
> > (3) You mention that ctypes probably doesn't
> > the GIL. I would guess that too, since it would
> > dangerous as I (vaguely) understand the GIL. But
> > the GIL have to be released in the Cextension or
> > it be release in the step just before I call the
> > extension from Python? I.e. is release on the
> > side possible? If not, I guess I will have to
> > over the numpy code as you suggest. If possible,
> > suppose the GIL must be enabled immediately on
> > from the C extension.
> You can't execute any python bytecodes without
> holding the GIL, so
> it's impossible for python code to release the GIL.
> But it would be
> perfectly possible, in principle, for SWIG, F2PY, or
> ctypes to put a
> "release the GIL" in their wrappers. This will be a
> problem for some
> functions - either ones that aren't reentrant, or
> ones that call back
> to python (though in principle it might be possible
> to reacquire the
> GIL for the duration of a callback). But for a
> typical C function that
> acts only on data you give it and that doesn't know
> anything about
> python, it should be safe to run it without the GIL
> engaged. It seems
> like f2py can actually do this for functions marked
> as threadsafe; I
> don't know about ctypes or SWIG.
Sounds like it's better to call those macros:
on the C side. Is that all that's needed? Then will
code like your handythread.py work with threads if f
calls a C extension that uses those macros? Or is
there more that needs to be done to set this up?
-- Lou Pecora, my views are my own.
Never miss a thing. Make Yahoo your home page.
More information about the SciPy-user