[SciPy-User] Porting code from IDL to Python - 'Common block' equivalent?
Fri Jul 23 02:01:28 CDT 2010
On Fri, Jul 23, 2010 at 2:26 AM, Charles R Harris
> On Wed, Jul 21, 2010 at 2:18 AM, David Andrews <email@example.com> wrote:
>> Hi All,
>> I suppose this might not strictly be a scipy type question, but I'll
>> ask here as I expect some of you might understand what I'm getting at!
>> I'm in the process of porting some code from IDL (Interactive Data
>> Language - popular in some fields of science, but largely nowhere
>> else) to Python. Essentially it's just plotting and analyzing time
>> series data, and so most of the porting is relatively simple. The one
>> stumbling block - is there an equivalent or useful replacement for the
>> "common block" concept in IDL available in Python?
>> Common blocks are areas of shared memory held by IDL that can be
>> accessed easily from within sub-routines. So for example, in our IDL
>> code, we load data into these common blocks at the start of a session,
>> and then perform whatever analysis on it. In this manner, we do not
>> have to continually re-load data every time we re-perform a piece of
>> analysis. They store their contents persistently, for the duration of
>> the IDL session. It's all for academic research purposes, so it's
>> very much 'try this / see what happens / alter it, try again' kind of
>> work. The loading and initial processing of data is fairly time
>> intensive, so having to reload at each step is a bit frustrating and
>> not very productive.
>> So, does anyone have any suggestions as to the best way to go about
>> porting this sort of behavior? Pickle seems to be one option, but
>> that would involve read/write to disk operations anyway? Any others?
> Depending on the sort of data you have, PyTables might be an option. I'm
> currently using it to store a 42 GB image data cube on disk and it works
> well for that. I can browse through an image and shift-click on a pixel to
> get a plot of the data associated with the pixel. It is quite fast. The data
> cube needs to be passed as an argument to the various functions that need
> the data, but that isn't much of a problem.
Chuck, just out of curiosity: what are the specs of your hardware and
which OS are you on ?
More information about the SciPy-User