[SciPy-dev] huge speed regression in loadmat from 0.6.0 to 0.7.0

[Robert Kern]

On Wed, Feb 11, 2009 at 14:09, Ryan May <rmay31@gmail.com> wrote:
> On Wed, Feb 11, 2009 at 2:03 PM, Scott David Daniels <Scott.Daniels@acm.org>
> wrote:
>>
>> Ryan May wrote:
>> > ... Well, here's a patch against gzipstreams.py that changes to add the
>> > chunks to a list and only add to the string at the very end. See if it
>> > helps your case.  If not, is there somewhere you can put the datafile so
>> > that we can test with it?
>> Well, in your patch, instead of:
>> @@ -95,11 +100,12 @@
>>              data = self.fileobj.read(n_to_fetch)
>>              self._bytes_read += len(data)
>>              if data:
>> -                self.data += self._unzipper.decompress(data)
>> +                self_data += self._unzipper.decompress(data)
>>              if len(data) < n_to_fetch: # hit end of file
>> -                self.data += self._unzipper.flush()
>> +                self_data += self._unzipper.flush()
>>                  self.exhausted = True
>>                  break
>> +        self.data += ''.join(self_data)
>>
>> Use:
>> @@ -95,11 +100,12 @@
>>              data = self.fileobj.read(n_to_fetch)
>>              self._bytes_read += len(data)
>>              if data:
>> -                self.data += self._unzipper.decompress(data)
>> +                self_data.append(self._unzipper.decompress(data))
>>              if len(data) < n_to_fetch: # hit end of file
>> -                self.data += self._unzipper.flush()
>> +                self_data.append(self._unzipper.flush())
>>                  self.exhausted = True
>>                  break
>> +        self.data += ''.join(self_data)
>>
>
> Yeah, you're right.  I thought += for lists just mapped to append, but
> apparently it appends other lists, but extends the list by other sequences.
> Weird.

Not weird at all. "x += y" should be the same as "x = x + y" except
for possible in-place modification, per the reference manual.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco