[SciPy-User] Running NumPy or SciPy scripts in the background on Windows
Thu Mar 1 09:54:53 CST 2012
Often when running long computations with NumPy or SciPy we want to "run
the task in the background". This is particularly important on Windows,
where a process that is greedy on CPU or RAM can almost make the system
unresponsive (e.g. the desktop seems to hang). On Unix there is the
"nice" command, but it is not available on Windows. We can set a process
priority with the Windows task manager, but that is of no help if the
system is unresponsive -- i.e. you cannot get to the task manager.
This is very simple to do with the Windows API (or pywin32). Here is how
a Python script (e.g. running NumPy or SciPy) can put itself in the
background using pywin32:
from win32process import (GetCurrentProcess,
These are the available flags for SetPriorityClass:
REALTIME_PRIORITY_CLASS ## absolute highest priority
## NB! Windows is not a RT OS,
## except Windows CE
NORMAL_PRIORITY_CLASS ## the normal priority
IDLE_PRIORITY_CLASS ## only execute when system is idle
Surprisingly many users of NumPy on Windows does not know this. I
thought we might put a "receipe" for doing this in the cookbook?
Sometimes we want to do this just for a thread, e.g. to keep an UI
responsive. We cannot control thread priorities with the Python stdlib.
Using pywin32, a Python thread that does this will put itself in the
background "relative to the priority class" for the process -- but not
relative to the rest of the system:
from win32api import GetCurrentThread
from win32process import THREAD_PRIORITY_IDLE, SetThreadPriority
Python interpreter does not attempt to schedule access to the GIL. That
is, a high-priority thread that releases the GIL might win it back, and
so get more time in the Python interpreter (at least on Python 2.x).
Similary a low-priority thread might more often miss the GIL battle. But
if the GIL is locked while an extension library (e.g. NumPy) is doing a
long computation, thread priority can be of no relevance. So this is
generally less useful than SetPriorityClass.
Here are the flags we can use for SetThreadPriority. Note that these are
relative to the "priority class" for the process (and complicated by the
GIL), not relative to the other processes on the system:
THREAD_PRIORITY_TIME_CRITICAL ## higher than 'highest'
More information about the SciPy-User