[SciPy-User] loading mat file in scipy

Alireza Alemi-Neissi alemi@sissa...
Wed Oct 21 17:03:30 CDT 2009


Thanks for all the comments.

I upgraded my python (2.6.3) and scipy (0.7.1) to the latest version.
Nevertheless, the mat file which loads in 10s in Matlab, loads in 612s
(~10min) in python. I also set the struct_as_array=True, but it did not
make  loading much faster ( 590s). I saved the mat file in version 5 of
matlab instead of version 7, again did not change the whole thing.

The file is too big (~120MB compressed). But I can tell you what is the
structure of it.

N is a <1 x 94 struct>
each N.Stim  is a <1 x 213 Cell>

each N.stim has a stuct. Let me write an example of first N and first Stim:

>>>N(1,1).Stim{1,1}

    NStimPerSet: 1
            Obj: [1x1 struct]
          Npres: 7
          times: {1x7 cell}
     SpikeTimes: {1x7 cell}
       PosInSeq: [4 9 14 20 6 4 9]
       TimeBins: [1x24 double]
     AvFireRate: [1x24 double]
      PlotColor: 'b'
             FR: [10.4167 0 0 0 10.4167 10.4167 0]
            AFR: 4.4643
            std: 5.5679
          ttest: [1x8 struct]


Please note that for all 94 member of N and all 213 member of Stim, this
fields are repeated.
To share the data, I have to get the approval of my supervisor. I will let
you know.

I am surprise that a 250Mb .mat files generated by ControlDesk (Dspace™)
can be loaded in python so fast. Is the nested structure is as compicated
as mine?

Disappointed with using loadmat, I am heading to use weave.inline to write
a code in C, loading mat file using the tools in mat.h header (hope it
does not get complicated!).
Loading to see seems straightforward. But, I have not figured out yet how
to  convert  the data  loaded in  C into python class (any clue?).
Do you have other ideas to make loadmat faster?

Thanks,
Alireza

> Hi,
>
>> I have recently tried to use scipy.io.loadmat() to load a mat file to
>> python.
>> The mat file is a complicated nested structure in matlab. It turned out
>> that it takes a long time (around 5 min) to load this data the size of
>> which is around 120MB.
>> On the other hand it take only a couple of seconds to load it in Matlab.
>> What is wrong with loadmat in scipy?
>
> What version of scipy do you have?  There was a performance bug I put
> in for 0.7 and removed for 0.7.1.
>
> If you are using 0.7.1, then maybe you can let me have a look at your
> file somehow - maybe by sharing a dropbox folder?
>
> Thanks a lot,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> SciPy-User@scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>



----------------------------------------------------------------
  SISSA Webmail https://webmail.sissa.it/
  Powered by SquirrelMail http://www.squirrelmail.org/



More information about the SciPy-User mailing list