[SciPy-User] Porting code from IDL to Python - 'Common block' equivalent?
Charles R Harris
Thu Jul 22 19:26:32 CDT 2010
On Wed, Jul 21, 2010 at 2:18 AM, David Andrews <firstname.lastname@example.org> wrote:
> Hi All,
> I suppose this might not strictly be a scipy type question, but I'll
> ask here as I expect some of you might understand what I'm getting at!
> I'm in the process of porting some code from IDL (Interactive Data
> Language - popular in some fields of science, but largely nowhere
> else) to Python. Essentially it's just plotting and analyzing time
> series data, and so most of the porting is relatively simple. The one
> stumbling block - is there an equivalent or useful replacement for the
> "common block" concept in IDL available in Python?
> Common blocks are areas of shared memory held by IDL that can be
> accessed easily from within sub-routines. So for example, in our IDL
> code, we load data into these common blocks at the start of a session,
> and then perform whatever analysis on it. In this manner, we do not
> have to continually re-load data every time we re-perform a piece of
> analysis. They store their contents persistently, for the duration of
> the IDL session. It's all for academic research purposes, so it's
> very much 'try this / see what happens / alter it, try again' kind of
> work. The loading and initial processing of data is fairly time
> intensive, so having to reload at each step is a bit frustrating and
> not very productive.
> So, does anyone have any suggestions as to the best way to go about
> porting this sort of behavior? Pickle seems to be one option, but
> that would involve read/write to disk operations anyway? Any others?
Depending on the sort of data you have, PyTables might be an option. I'm
currently using it to store a 42 GB image data cube on disk and it works
well for that. I can browse through an image and shift-click on a pixel to
get a plot of the data associated with the pixel. It is quite fast. The data
cube needs to be passed as an argument to the various functions that need
the data, but that isn't much of a problem.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-User