[Numpy-svn] [numpy/numpy] cfae01: BUG: fix loading large npz files (fixes #2922)

GitHub noreply@github....
Wed Jun 12 09:41:39 CDT 2013


  Branch: refs/heads/master
  Home:   https://github.com/numpy/numpy
  Commit: cfae0143b436c3296eebe71e2dd730625dcaae95
      https://github.com/numpy/numpy/commit/cfae0143b436c3296eebe71e2dd730625dcaae95
  Author: Bartosz Telenczuk <muchatel@poczta.fm>
  Date:   2013-06-12 (Wed, 12 Jun 2013)

  Changed paths:
    M numpy/lib/npyio.py
    M numpy/lib/tests/test_io.py

  Log Message:
  -----------
  BUG: fix loading large npz files (fixes #2922)


  Commit: b69c48d34d6b6d9be01f37bd5117e946e2556df8
      https://github.com/numpy/numpy/commit/b69c48d34d6b6d9be01f37bd5117e946e2556df8
  Author: Alex Ford <fordas@uw.edu>
  Date:   2013-06-12 (Wed, 12 Jun 2013)

  Changed paths:
    M numpy/lib/format.py

  Log Message:
  -----------
  Chunk reads in format.read_array.

Maximum data size limitations in the crc32 module cause errors when
reading more than 2 ** 32 bytes from gzip streams. Work around this
issue when reading large arrays from npz files by chunking reads to
256mb.

This appears to resolve bug #2922.


  Commit: 7c4e9e14c473060595271a856b307bbc04f1c7bb
      https://github.com/numpy/numpy/commit/7c4e9e14c473060595271a856b307bbc04f1c7bb
  Author: Bartosz Telenczuk <muchatel@poczta.fm>
  Date:   2013-06-12 (Wed, 12 Jun 2013)

  Changed paths:
    M doc/release/1.8.0-notes.rst
    M numpy/lib/format.py
    M numpy/lib/npyio.py
    M numpy/lib/tests/test_io.py

  Log Message:
  -----------
  adjust the optimal IO buffer size for npz files


  Commit: f7ea47401ebd4cee5853c62ba3c849c9fdeaea46
      https://github.com/numpy/numpy/commit/f7ea47401ebd4cee5853c62ba3c849c9fdeaea46
  Author: Charles Harris <charlesr.harris@gmail.com>
  Date:   2013-06-12 (Wed, 12 Jun 2013)

  Changed paths:
    M doc/release/1.8.0-notes.rst
    M numpy/lib/format.py
    M numpy/lib/npyio.py
    M numpy/lib/tests/test_io.py

  Log Message:
  -----------
  Merge pull request #2942 from btel/npz_optimize

BUG: Fix loading npz files >2GB on 64bit systems


Compare: https://github.com/numpy/numpy/compare/d4b4ff038d53...f7ea47401ebd


More information about the Numpy-svn mailing list