[SciPy-User] Running NumPy or SciPy scripts in the background on Windows

Sturla Molden sturla@molden...
Thu Mar 1 09:54:53 CST 2012


Often when running long computations with NumPy or SciPy we want to "run 
the task in the background". This is particularly important on Windows, 
where a process that is greedy on CPU or RAM can almost make the system 
unresponsive (e.g. the desktop seems to hang). On Unix there is the 
"nice" command, but it is not available on Windows. We can set a process 
priority with the Windows task manager, but that is of no help if the 
system is unresponsive -- i.e. you cannot get to the task manager.

This is very simple to do with the Windows API (or pywin32). Here is how 
a Python script (e.g. running NumPy or SciPy) can put itself in the 
background using pywin32:


     from win32process import (GetCurrentProcess,
        IDLE_PRIORITY_CLASS, SetPriorityClass)
     SetPriorityClass(GetCurrentProcess(), IDLE_PRIORITY_CLASS)


These are the available flags for SetPriorityClass:


REALTIME_PRIORITY_CLASS       ## absolute highest priority
                               ## NB! Windows is not a RT OS,
                               ## except Windows CE
ABOVE_NORMAL_PRIORITY_CLASS
NORMAL_PRIORITY_CLASS         ## the normal priority
BELOW_NORMAL_PRIORITY_CLASS
IDLE_PRIORITY_CLASS           ## only execute when system is idle


Surprisingly many users of NumPy on Windows does not know this. I 
thought we might put a "receipe" for doing this in the cookbook?




Sometimes we want to do this just for a thread, e.g. to keep an UI 
responsive. We cannot control thread priorities with the Python stdlib. 
Using pywin32, a Python thread that does this will put itself in the 
background "relative to the priority class" for the process -- but not 
relative to the rest of the system:

     from win32api import GetCurrentThread
     from win32process import THREAD_PRIORITY_IDLE, SetThreadPriority
     SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_IDLE)

Python interpreter does not attempt to schedule access to the GIL. That 
is, a high-priority thread that releases the GIL might win it back, and 
so get more time in the Python interpreter (at least on Python 2.x). 
Similary a low-priority thread might more often miss the GIL battle. But 
if the GIL is locked while an extension library (e.g. NumPy) is doing a 
long computation, thread priority can be of no relevance. So this is 
generally less useful than SetPriorityClass.

Here are the flags we can use for SetThreadPriority. Note that these are 
relative to the "priority class" for the process (and complicated by the 
GIL), not relative to the other processes on the system:

THREAD_PRIORITY_TIME_CRITICAL ## higher than 'highest'
THREAD_PRIORITY_HIGHEST
THREAD_PRIORITY_ABOVE_NORMAL
THREAD_PRIORITY_NORMAL
THREAD_PRIORITY_BELOW_NORMAL
THREAD_PRIORITY_LOWEST
THREAD_PRIORITY_IDLE





Sturla




More information about the SciPy-User mailing list