[SciPy-user] problem with using scipy.io.mio on Windows

Herb Schilling Herbert.W.Schilling at grc.nasa.gov
Thu Jan 17 07:48:09 CST 2002


Hi Travis,

>
>Note, fopen is defined right under scipy.io (you shouldn't have to import
>mio).
>
>so after an
>
>import scipy.io
>
>scipy.io.fopen  should work


Thanks for the tip!

>
>>
>
>For some reason the C-code is not reading as many bytes as was requested.
>This warning just alerts you to that fact.
>
>One problem I've seen is byte order issues.  What is the byte-order on an
>SGI?

Big endian.

>  It looks like you are opening the file with "big-endian" so that
>shouldn't be it.  Can you send me an example file that I could try to read
>on Linux?

Sure. I have attached a file. Sorry for the size. Floating point data 
files don't compress well.

>
>One test you could do is to run fileHandle.tell() to see where the
>file-pointer is after each read.  This should tell you how you are
>progressing through the file.


Excellent idea! I just tried that and the results are very strange. 
Calling tell affects the results!
Also it is very strange that after reading in one integer that the 
position leaps to 3588 ! Some of the other positions don't make sense 
either.

Even the values in the warnings don't make sense:

	numpyio: Warning: 7515 bytes requested, 12 bytes read.

This came when I tried to read 501 x 15 floats which should have been 
7515 * 4 bytes, not 7515 bytes.


Below are the contents of three files:

	1. The script I use to read the PLOT3D file I have attached.

	2. The output of the script when I am NOT calling tell

	3. The output of the script when I am calling tell


===============================
script - note that I comment/uncomment a line in the whatIsPosition 
function to change
	my call to tell
===============================
import sys

def whatIsPosition( fileHandle, whatWasJustRead ):
   pass
   #print "After " + whatWasJustRead + ", position is " , fileHandle.tell()


def readPlot3DBinaryToBlocks( filePath ):
   '''Read in a binary PLOT3D file to a set of Blocks.'''

   import scipy.io

   # Open file 
   fileHandle = scipy.io.fopen( filePath, 'r' , format = 'b' )
   print "fileSize is " , fileHandle.size()
   whatIsPosition( fileHandle , "fopen" )

   # read in number of blocks
   numberOfBlocks = fileHandle.fread( 1 , 'l' )
   whatIsPosition( fileHandle , "reading number of blocks" )
   print "number of blocks is " , numberOfBlocks

   # Read in the size of each block
   imaxes = []
   jmaxes = []
   for iBlock in range( numberOfBlocks ) :
     imax = fileHandle.fread( 1 , 'l' )
     jmax = fileHandle.fread( 1 , 'l' )
     whatIsPosition( fileHandle , "reading imax and jmax" )
     print "for block # " , iBlock, "imax and jmax are " , imax , jmax
     imaxes.append( imax )
     jmaxes.append( jmax )

   # read in the X and Y values, first comes block 1 X, then block 1 
Y, then block 3 X, then block 3 Y
   blocks = []
   for iBlock in range( numberOfBlocks ) :
     imax = imaxes[ iBlock ]
     jmax = jmaxes[ iBlock ]

     # Note that we do want the transposed form
     xGrid = fileHandle.fread(  ( jmax, imax ) , 'f' , 'f' )
     yGrid = fileHandle.fread(  ( jmax, imax ) , 'f' , 'f' )


     print "for block # " , iBlock, "xGrid[ 0: 3 , 0:3 ] is " , xGrid[ 
0: 3 , 0:3 ]
     print "for block # " , iBlock, "yGrid[ 0: 3 , 0:3 ] is " , yGrid[ 
0: 3 , 0:3 ]

     whatIsPosition( fileHandle , "reading xGrid and yGrid" )

   return


print sys.argv[1]
blocks = readPlot3DBinaryToBlocks( sys.argv[1] )


===============================
The output when I am NOT calling tell
===============================
binary.p3d
fileSize is  292604
number of blocks is  5
for block #  0 imax and jmax are  501 15
for block #  1 imax and jmax are  163 17
for block #  2 imax and jmax are  156 59
for block #  3 imax and jmax are  427 20
for block #  4 imax and jmax are  427 20
for block #  0 xGrid[ 0: 3 , 0:3 ] is  [[ 15.99999905  13.77125454 
11.87302971]
  [  4.51538134   3.98979473   3.54215217]
  [  6.70775032   5.85703707   5.13248396]]
for block #  0 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  1 xGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  1 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  2 xGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  2 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  3 xGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  3 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  4 xGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
for block #  4 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]


===============================
The output when I am NOT calling tell
===============================
binary.p3d
fileSize is  292604
After fopen, position is  0
After reading number of blocks, position is  3588
number of blocks is  5
After reading imax and jmax, position is  3596
for block #  0 imax and jmax are  501 15
After reading imax and jmax, position is  3604
for block #  1 imax and jmax are  163 17
After reading imax and jmax, position is  3612
for block #  2 imax and jmax are  156 59
After reading imax and jmax, position is  3620
for block #  3 imax and jmax are  427 20
After reading imax and jmax, position is  3628
for block #  4 imax and jmax are  427 20
for block #  0 xGrid[ 0: 3 , 0:3 ] is  [[ 1.59999990e+001 
1.37712545e+001  1.18730297e+001]
  [ 3.54215217e+000  3.16090393e+000  1.64749781e+021]
  [ 5.13248396e+000  4.51538134e+000  3.98979473e+000]]
for block #  0 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
After reading xGrid and yGrid, position is  32768
for block #  1 xGrid[ 0: 3 , 0:3 ] is  [[-0.01891858 -0.01973816 -0.02066956]
  [-0.03023803 -0.03107315 -0.03175394]
  [-0.02180799 -0.02299393 -0.02415288]]
for block #  1 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
After reading xGrid and yGrid, position is  40960
for block #  2 xGrid[ 0: 3 , 0:3 ] is  [[-0.02406131 -0.02338937 -0.02269458]
  [-0.00455854 -0.00379783 -0.00291636]
  [ 0.01606247  0.01684676  0.01759628]]
for block #  2 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
After reading xGrid and yGrid, position is  73728
for block #  3 xGrid[ 0: 3 , 0:3 ] is  [[-0.03620248 -0.03564961 -0.03520766]
  [-0.03422029 -0.03353022 -0.03296501]
  [-0.03248635 -0.03199605 -0.03620248]]
for block #  3 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
After reading xGrid and yGrid, position is  106496
for block #  4 xGrid[ 0: 3 , 0:3 ] is  [[ 0.96958733  0.97537327  0.98071378]
  [ 1.01128232  1.02077925  1.03193545]
  [ 1.12465644  1.1539247   1.18828964]]
for block #  4 yGrid[ 0: 3 , 0:3 ] is  [[ 0.  0.  0.]
  [ 0.  0.  0.]
  [ 0.  0.  0.]]
After reading xGrid and yGrid, position is  139264






>
>Sorry I can't help further.

On the contrary, you have given me a lot of help! I was ready to 
abandon scipy completely and use SWIG to wrap some C code to read 
PLOT3D files but I thought it would easier to use scipy and it was on 
UNIX !

Thanks!
-- 
---------------------------------
Herb Schilling
NASA Glenn Research Center
Cleveland, OH 44135
Herbert.W.Schilling at grc.nasa.gov

It is not enough to run, one must start in time. - teabag
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.scipy.org/pipermail/scipy-user/attachments/20020117/0612fac3/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: %binary.p3d.gz
Type: application/applefile
Size: 123 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/scipy-user/attachments/20020117/0612fac3/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: binary.p3d.gz
Type: application/octet-stream
Size: 252494 bytes
Desc: not available
Url : http://projects.scipy.org/pipermail/scipy-user/attachments/20020117/0612fac3/attachment.obj 


More information about the SciPy-user mailing list