[Numpy-discussion] reading 10 bit raw data into an array

Travis Oliphant oliphant.travis@ieee....
Mon Jul 30 11:54:26 CDT 2007


Danny Chan wrote:
> Hi all!
> I'm trying to read a data file that contains a raw image file. Every 
> pixel is assigned a value from 0 to 1023, and all pixels are stored from 
> top left to bottom right pixel in binary format in this file. I know the 
> width and the height of the image, so all that would be required is to 
> read 10 bits at a time and store it these as an integer. I played around 
> with the fromstring and fromfile function, and I read the documentation 
> for dtype objects, but I'm still confused. It seems simple enough to 
> read data in a format with a standard bitwidth, but how can I read data 
> in a non-standard format. Can anyone help?
> 

This kind of bit-manipulation must be done using bit operations on 
standard size data types even in C.  The file reading and writing 
libraries use bytes as their common denominator.

I would read in the entire image into a numpy array of unsigned bytes 
and then use slicing, masking, and bit-shifting to take 5 bytes at a 
time and convert them to 4 values of a 16-bit unsigned image.

Basically, you would do something like

# read in entire image into 1-d unsigned byte array
# create 16-bit array of the correct 2-D size
# use flat indexing to store into the new array
#   new.flat[::4] = old[::5] + bitwise_or(old[1::5], MASK1b) << SHIFT1b
#   new.flat[1::4] = bitwise_or(old[1::5], MASK2a) << SHIFT2a
                     + bitwise_or(old[2::5], MASK2b) << SHIFT2b

#   etc.


The exact MASKS and shifts to use is left as an exercise for the reader :-)


-Travis


More information about the Numpy-discussion mailing list