[SciPy-User] FreeImage <-> numpy IO wrappers

Zachary Pincus zachary.pincus@yale....
Tue Oct 19 13:16:02 CDT 2010


On Oct 19, 2010, at 1:59 PM, Sebastian Haase wrote:

> On Tue, Oct 19, 2010 at 6:10 PM, Zachary Pincus <zachary.pincus@yale.edu 
> > wrote:
>> Hi Sebastien,
>>
>> Thanks -- I hope this winds up being useful. Stéfan tried out this  
>> code a
>> while ago and ran into a segfault loading a color jpeg that I was  
>> never able
>> to reproduce... perhaps something to do with the fact that his
>> python/FreeImage were 64-bit. Anyhow, beware that caveat.
>>
>>> Here are some comment and questions:
>>>
>>> 1) I would rename it to freeImage.py - or alike
>>
>> Yeah, sure, that's reasonable. Below is a new version of the code  
>> that's a
>> package that you import as "FreeImage"...
>>
>>> 2) Compiling freeimage from source went really well. No configure,
>>> simple a "make" and it ran through. Except in my version
>>> freeimage-3.14.1 I had to add "#include <string.h>" to file
>>> ImathMatrix.h so that it would accept the use of memset(...)
>>
>> Huh... that's odd. Good to know!
>>
>>> 3) you are changing the FreeImage convention of 0,0 being bottom- 
>>> left
>>> to 0,0 being top-left -- to make it more like "all other" image
>>> software.   I actually come from UCSF where we used the MRC format
>>> having 0,0 being left-bottom.  How strong do you feel about this ?
>>
>> Pretty much all of the common basic image formats (TIFF, PNG, JPG  
>> and the
>> like) have 0,0 as top-left, so I tried to make it so that images  
>> loaded
>> would index just the same as they would in any other image viewer  
>> like
>> MetaMorph or ImageJ or Photoshop, etc. This explains the 0,0-as-top- 
>> left as
>> well as your question 5 below about the striding.
>>
>> Also, the image formats I'm most familiar with are by-and-large  
>> stored
>> on-disk in scanline order from top to bottom, then additional image  
>> planes
>> (also in scanline order). Which corresponds to fortran-order (fast  
>> axis
>> first) xy[zt] in memory, with 0,0 at the top-left. Which is also  
>> what most
>> libraries expect when passed an memory region that's supposed to be  
>> an image
>> (e.g. OpenGL, other UI libs, etc.), so I tried to make the output  
>> conform to
>> the most usual "expected" memory pattern.
>>
>> One strange thing crops up with color images. Usually RGB(A) pixels  
>> are
>> stored next to eachother, so the fortran-order memory format is cxyzt
>> (again, this is what external libraries expect). But a quick .view()
>> operation with a structured RGB(A) dtype makes this work more  
>> "naturally"
>> (if desired).
>>
>>> 4) For the return numpy-array you explicitly call copy() -  I guess
>>> this is needed because one has to call FreeImage's unload() ,  
>>> right ?
>>
>> Yes, this is the case.
>>
>> I think there's some way in the numpy C-api (or perhaps the new py3  
>> buffer
>> api?) to specify a function pointer to be called when an array  
>> needs to be
>> freed (if the array is constructed around some external chunk of  
>> memory). So
>> if some numpy hacker who knows the C api better than I could  
>> comment on how
>> one might do this, it could be arranged such that FreeImage_unload  
>> is called
>> by numpy, so that we don't need to copy the array and then manually  
>> unload
>> the image.
>>
>>> 5) you define the array strides to have the pitch (that is, the line
>>> width) last -- this is somewhat against the C-convention of having  
>>> the
>>> fast axis last. Obviously you did this, to get arrays with indices  
>>> i,j
>>> having x,y order rather than y,x -- how strong do you feel about  
>>> this
>>> ? I accepted at some point that the fast (x) coordinate would be  
>>> last
>>> and thus always write coordinates as y,x.
>>> (In 3D this becomes semi-naturally z,y,x  rather then z,x,y - BTW  
>>> - )
>>
>> See above... given that images are usually fortran-order on disk  
>> and that
>> most external libraries expect them to be that way too, I think  
>> that this is
>> the most reasonable. It's just that PIL has been so broken for so  
>> long that
>> people on Python are used to indexing images as i[y,x] instead of  
>> i[x,y].
>> But fortran-order arrays are just as natural as C-order, and  
>> i[x,y,z,t] is
>> also as natural, if not more so, than i[t,z,y,x].
>>
>> Again, the code is barely a few hundred lines, so feel free to  
>> modify to the
>> conventions that are most natural for your application! But I think  
>> this is
>> the best general-case approach.
>>
>>> Do you happen to have os-x binaries of libfreeimage.dylib ?
>>
>> I do -- 32-bit only though. (Pretty easy to build...) Do you want  
>> me to send
>> the dylib?
>>
>> Attached is the latest version of my code, which has a few new bug  
>> fixes
>> from the previous version. (Can read palletized grey-scale images,  
>> for
>> example.) Just drop whatever .dylib, .dll, or .so FreeImage shared  
>> library
>> you've got into the directory with image.py and the setup.py script  
>> will
>> detect it and install it alongside the python code. Then just do  
>> "import
>> FreeImage" and you're good to go.
>>
>> Zach
>>
> I really don't want to edit your python file - even though I
> appreciate the fact that it is quite small.
> I would rather write smaller wrapper functions to get the memory  
> order I need.
> Another way would be to add some sort of mem_layout flag to support
> the four possible permutations c<->fortran and bottom<->top at the
> lowest level and prevent an extra copy.

The good news is that this can all be done without any copying needed:
i.T will switch from i[x,y] (fortran order) to i[y,x] (C order), just  
by reversing the shape and strides, without copying data. Likewise,  
i[::-1] will give a top-to-bottom reversed array also without any  
copying, by using a negative stride. Sure, you'll need to copy the  
data before handing it to something that expects a contiguous array,  
but most numpy operations will work just fine. Though, honestly,  
compared to the latency of reading images from disk, probably an extra  
copy wouldn't even be perceptible in terms of performance.

> I had never heard that image data (file formats, OpenGL, etc.) was not
> ordered in y,x (C-order, i.e. x being fast) memory layout - I though
> the top vs. bottom origin was the only "uncertainty" - well, there
> maybe more FORTRAN around than I thought ...

It's not to do so much with FORTRAN as the fact that images are  
typically blitted in "raster order" to the screen(buffer): left-to- 
right, top-to-bottom. Probably doesn't matter much anymore, but maybe  
that's what was fastest in the early days? Anyway, yeah, most image  
formats store things in scanlines, unless there's chunked compression,  
or interleaving, etc.

Zach

PS. consider the code I sent to be in the public domain. Do with it  
what you will.

> I will take a look at your update tomorrow.
> - Sebastian
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user



More information about the SciPy-User mailing list