[Numpy-discussion] Could numpy.fromfile() work with gzip-compressed files ?

[bruno Piguet]

Hi all,

  I've got some code which basically does :

  f  = open (path, 'rb')
  header = f.read (some_length)
  data = np.fromfile (f, dtype= some_type, count=-1)

  In order to process compressed files, I switched the open sequence to :

  if (plain):
      f  = open (path, 'rb')
  else:
      f  = gzip.open (path, 'rb')

  Then, the header reading is OK, since the gzip.file class has a "read "
method :
  hdr = f.read (some_length)

  But I've got problems with np.fromfile, which says :
IOError: first argument must be an open file

   I guess that fromfile() uses methods other than file.read(), while
GzipFile does not simulates all the methods of a file object (readinto() and
truncate() are listed the doc), so making fromfile() work with compressed
file may not be straightforward.

   I had a try at StringIO() :
  data = np.fromfile (StringIO.StringIO(f.read()), dtype= some_type,
count=-1)
  but it doesn't work better.

So, I've got two questions :
 - Could numpy.fromfile() work with gzip-compressed files, as some other
file functions do ?
 - I'm thinking of using something like  np.array(f.read(),
dtype=some_type). Any better solution ? With such a simple solution, maybe
my error was to try to use fromfile() at the beginning ?

Bruno.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/numpy-discussion/attachments/20100726/8c36fe7d/attachment.html