[AstroPy] Moving Ahead with Raw Image Conversion

Wayne Watson sierra_mtnview@sbcglobal....
Tue Apr 14 07:18:57 CDT 2009


Hi, I'm headed back to sleep. I'll just mention the circular picture is 
the view of the sky from my fisheye camera. The perimeter is the horizon 
with trees. There are a few clouds in the sky, bands. later ... zzzzzzzzz

Peter Erwin wrote:
> Hi Wayne,
>
> Anne Archibald pointed out a much more concise and elegant way to read in
> and convert the raw file, using a function in Numpy.  So you could 
> replace the
> following original code:
>
>> # get raw data from .raw file:
>> raw_file=open('sent_internal.raw','rb')
>> raw_image=raw_file.read()
>> raw_file.close()
>>
>> # convert raw data to Python list of integers (1-D):
>> rawim_intarray = []
>> for x in raw_image:
>>   # convert byte to integer ('B' = treat data as single unsigned byte)
>>   newint = struct.unpack('B', x)[0]
>>   rawim_intarray.append(newint)
>>
>> # convert Python list of integers into numpy array of integers (note: 
>> still 1-D at this point)
>> rawim_numpy = numpy.array(rawim_intarray)
>
> with this:
>
> rawim_numpy = numpy.fromfile('sent_internal.raw', dtype='B')
>
> [note: dtype='B' is identical to dtype=numpy.uint8, which Anne used in 
> her example]
>
> and then continue as before.
>
> This has the added benefit of producing a smaller output file (300 KB vs
> 1.2 MB), since the numpy array is now explicitly in 8-bit-integer form
> (rather than the 32-bit-integer form that my code created), and so
> Pyfits will automatically save it as 8-bit integers (you can check this
> by looking at the header keyword BITPIX).
>
>
> You're right, the raw image is a sequence of (standard) 8-bit bytes.  
> Well, all
> computer files are streams of bytes; the question is how to interpret 
> them.  Since
> you said earlier that it was supposed to be 8-bit values, the idea is 
> then to
> interpret each individual byte as an unsigned "integer" value (values 
> from 0 to 255);
> that's what the "dtype='B'" (or "dtype=numpy.uint8") option passed to 
> numpy.fromfile()
> does.  (My previous code did the same thing, but then assigned each 
> value to a
> standard Python integer, which is 4 bytes long; perfectly workable, 
> but a bit
> wasteful!).
>
> Anyway, when I do this, I can see an image in DS9 (a circular aperture 
> with ragged
> edges and an overall horizontal gradient inside the aperture).  So I'm 
> pretty sure
> it's working.  (I've attached a jpeg saved from DS9 to show what the 
> image looks like.)
>
> I don't think I can help you with this "ccdsoft" program; I'm not 
> familiar with it.
> If it's a Windows program, then it may expect all FITS files to end in 
> ".fit" instead
> of ".fits", just because Windows traditionally expects files to have 
> 3-letter extensions.
>
>
> As for overwriting an existing FITS file, try this:
>
> hdu.writeto("existing_file_name.fit", clobber=True)
>
> cheers,
>
> Peter
>
>
>
> ------------------------------------------------------------------------
>
>
>
>
> On Apr 14, 2009, at 6:28 AM, Wayne Watson wrote:
>
>> Hi, Peter. It looks like I confused a few people with the hex. I had 
>> removed it, but only after I placed the code in the msg. I was 
>> playing with hex in the event I might need it somehow. int was fine 
>> for my experimental purposes.
>>
>> Here's your code with a minor correction. I changed this line:
>>  rawim_numpy_2d = numpy.reshape(rawim_numpy,(480, 640))
>> The difference is rawmin_numpy.
>> I added a print at the end, and threw in two lines that attempt to 
>> put header info into the final file.  Continuing below the code.
>>
>> =====================Start===============
>> import struct, numpy, pyfits   # struct is part of the main Python 
>> library
>>
>> # get raw data from .raw file:
>> raw_file=open('sent_internal.raw','rb')
>> raw_image=raw_file.read()
>> raw_file.close()
>>
>> # convert raw data to Python list of integers (1-D):
>> rawim_intarray = []
>> for x in raw_image:
>>   # convert byte to integer ('B' = treat data as single unsigned byte)
>>   newint = struct.unpack('B', x)[0]
>>   rawim_intarray.append(newint)
>>
>> # convert Python list of integers into numpy array of integers (note: 
>> still 1-D at this point)
>> rawim_numpy = numpy.array(rawim_intarray)
>>
>> # reshape numpy array into 2D form, using our knowledge of the 
>> original image's
>> # x and y sizes (kudos to Megan Sosey for pointing out how to do this)
>> rawim_numpy_2d = numpy.reshape(rawim_numpy,(480, 640))
>>
>> # create Pyfits header-data unit:
>> hdu = pyfits.PrimaryHDU(rawim_numpy_2d)
>> hdu.header.update('LATOBS',"32:11:56")
>> hdu.header.update('LNGOBS',"120:00:00")
>>
>> # (make any modifications to the header you might want to... e.g., 
>> see Megan Sosey's
>> # email of April 8 for examples)
>>
>> # ... and save the data to disk as a FITS file:
>> hdu.writeto("test.fits")
>> print "Finished. See test.fits"
>> ===============End code============
>>
>> I'm not sure what the 'byte' stuff is about. It may be that it's not 
>> needed. I think the original 'raw' file is byte oriented--rather than 
>> int or whatever. Anyway,  I don't seem to have succeeded in getting 
>> header info into the file. Sort of. If I use DS9, I can see the image 
>> and the fits header with what I coded into it. Here's what I see with 
>> ds9.
>>
>> *However, with my ccdsoft program, te image is black and the header 
>> is not the same. Whoops. Interesting. I changed the file to test.fit 
>> instead of test.fits, and the header info is there, but the image is 
>> still black. The dimension are the same, 640x480. I'm attaching 
>> test-ww.fits. I just renamed my test.fits. I think I'll check with 
>> The Bisque (ccdsoft) to see what format they produce and use for the 
>> raw image.
>> Question. When I execute the program a second time, the 
>> *hdu.writeto("test.fits") *will not write over the old file. Is there 
>> an option to force the write?
>>
>> Well, this is all good progress. **  *
>>
>> -- 
>>
>>          Wayne Watson (Watson Adventures, Prop., Nevada City, CA)
>>
>>            (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
>>             Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet
>>          All the neutrons, and protons in the human body occupy
>>          a cube whose side is 5.52*10**-6 meters (tiny!). That
>>          adds up to a 150 pound person. It's not a surprise that
>>          we are mostly space. (Calculation by WTW)
>
> =============================================================
> Peter Erwin                   Max-Planck-Insitute for Extraterrestrial
> erwin@mpe.mpg.de              Physics, Giessenbachstrasse
> tel. +49 (0)89 30000 3695     85748 Garching, Germany
> fax  +49 (0)89 30000 3495     http://www.mpe.mpg.de/~erwin
>
>
>

-- 
           Wayne Watson (Watson Adventures, Prop., Nevada City, CA)

             (121.015 Deg. W, 39.262 Deg. N) GMT-8 hr std. time)
              Obz Site:  39° 15' 7" N, 121° 2' 32" W, 2700 feet  

           All the neutrons, and protons in the human body occupy
           a cube whose side is 5.52*10**-6 meters (tiny!). That
           adds up to a 150 pound person. It's not a surprise that
           we are mostly space. (Calculation by WTW)
 



More information about the AstroPy mailing list