[NumPy-Tickets] [NumPy] #2178: regression: closes not-owned handle

[NumPy Trac]

#2178: regression: closes not-owned handle
------------------------+---------------------------------------------------
 Reporter:  yarikoptic  |       Owner:  somebody   
     Type:  defect      |      Status:  new        
 Priority:  normal      |   Milestone:  Unscheduled
Component:  Other       |     Version:  1.6.1      
 Keywords:              |  
------------------------+---------------------------------------------------

Comment(by yarikoptic):

 a bit of the stack/trace for real-world example where in
 numpy.lib.npyio.NpzFile.close() (called from __del__) it would close the
 handle which it doesn't own... may be because somewhere it first assigned
 the same original FID to the temporary zipfile:

 {{{
 $> PYTHONPATH=$PWD:/home/yoh/proj/numpy nosetests -s -v
 test/unittests/test_files.py:test_NumpyBinaryFile
 /usr/bin/nosetests:5: UserWarning: Module paste was already imported from
 None, but /usr/lib/python2.7/dist-packages is being added to sys.path
   from pkg_resources import load_entry_point
 /usr/lib/python2.7/dist-packages/tables/__init__.py:59: RuntimeWarning:
 numpy.dtype size changed, may indicate binary incompatibility
   from tables.utilsExtension import getPyTablesVersion, getHDF5Version
 /usr/lib/python2.7/dist-packages/tables/file.py:45: RuntimeWarning:
 numpy.dtype size changed, may indicate binary incompatibility
   from tables import hdf5Extension
 /usr/lib/python2.7/dist-packages/tables/link.py:32: RuntimeWarning:
 numpy.dtype size changed, may indicate binary incompatibility
   from tables import linkExtension
 /usr/lib/python2.7/dist-packages/tables/table.py:40: RuntimeWarning:
 numpy.dtype size changed, may indicate binary incompatibility
   from tables import tableExtension
 /usr/lib/python2.7/dist-packages/tables/index.py:45: RuntimeWarning:
 numpy.dtype size changed, may indicate binary incompatibility
   from tables import indexesExtension
 unittests.test_files.test_NumpyBinaryFile ... (/home/yoh/deb/gits/pkg-
 exppsy/pynn/test/unittests/test_files.py:107):  test_NumpyBinaryFile
 107     metadata_ = nbf.get_metadata()
 (Pydb) print nbf.fileobj
 <open file 'tmp.npz', mode 'r' at 0x2c2c150>
 (Pydb) b /usr/lib/python2.7/zipfile.py:1157
 Breakpoint 1 set in file /usr/lib/python2.7/zipfile.py, line 1157.
 (Pydb) c
 (/usr/lib/python2.7/zipfile.py:1157):  close
 1157         if self.fp is None:
 (Pydb) print self.fp
 <open file 'tmp.npz', mode 'r' at 0x2c2c150>
 (Pydb) bt
 -> 0 close(self=<zipfile.ZipFile instance at 0x2d6aa28>) called from file
 '/usr/lib/python2.7/zipfile.py' at line 1157
 ## 1 close(self=<numpy.lib.npyio.NpzFile object at 0x2dd3a90>) called from
 file '/home/yoh/proj/numpy/numpy/lib/npyio.py' at line 212
 ## 2 __del__(self=<numpy.lib.npyio.NpzFile object at 0x2dd3a90>) called
 from file '/home/yoh/proj/numpy/numpy/lib/npyio.py' at line 220
 ## 3 test_NumpyBinaryFile() called from file '/home/yoh/deb/gits/pkg-
 exppsy/pynn/test/unittests/test_files.py' at line 107
 ## 4 runTest(self=unittests.test_files.test_NumpyBinaryFile) called from
 file '/usr/share/pyshared/nose/case.py' at line 197
 ## 5 run(self=unittests.test_files.test_NumpyBinaryFile,
 result=<nose.result.TextTestResult run=1 errors=0 f...) called from file
 '/usr/lib/python2.7/unittest/case.py' at line 327
 ## 6 __call__(self=unittests.test_files.test_NumpyBinaryFile,
 *args=(<nose.result.TextTestResult run=1 errors=0 f...) called from file
 '/usr/lib/python2.7/unittest/case.py' at line 391
 ## 7 runTest(self=Test(unittests.test_files.test_NumpyBinaryFile),
 result=<nose.result.TextTestResult run=1 erro...) called from file
 '/usr/share/pyshared/nose/case.py' at line 151
 ## 8 run(self=Test(unittests.test_files.test_NumpyBinaryFile),
 result=<nose.result.TextTestResult run=1 erro...) called from file
 '/usr/share/pyshared/nose/case.py' at line 133
 ## 9 __call__(self=Test(unittests.test_files.test_NumpyBinaryFile),
 *arg=(<nose.result.TextTestResult run=1 error...) called from file
 '/usr/share/pyshared/nose/case.py' at line 45
 ## 10 run(self=<nose.suite.ContextSuite context=unittests.test_files>,
 result=<nose.result.TextTestResult run...) called from file
 '/usr/share/pyshared/nose/suite.py' at line 223
 ## 11 __call__(self=<nose.suite.ContextSuite
 context=unittests.test_files>, *arg=(<nose.result.TextTestResult run=...)
 called from file '/usr/share/pyshared/nose/suite.py' at line 176
 ## 12 run(self=<nose.suite.ContextSuite context=unittests.test_files>,
 result=<nose.result.TextTestResult run...) called from file
 '/usr/share/pyshared/nose/suite.py' at line 223
 ## 13 __call__(self=<nose.suite.ContextSuite
 context=unittests.test_files>, *arg=(<nose.result.TextTestResult run=...)
 called from file '/usr/share/pyshared/nose/suite.py' at line 176
 ## 14 run(self=<nose.core.TextTestRunner object at 0x2108fd0>,
 test=<nose.suite.ContextSuite context=unittest...) called from file
 '/usr/share/pyshared/nose/core.py' at line 61
 ## 15 runTests(self=<nose.core.TestProgram object at 0x1cf19d0>) called
 from file '/usr/share/pyshared/nose/core.py' at line 197
 ## 16 __init__(self=<nose.core.TestProgram object at 0x1cf19d0>,
 module=None, defaultTest='.', argv=['/usr/bin/nos...) called from file
 '/usr/lib/python2.7/unittest/main.py' at line 95
 ## 17 __init__(self=<nose.core.TestProgram object at 0x1cf19d0>,
 module=None, defaultTest='.', argv=None, testRunn...) called from file
 '/usr/share/pyshared/nose/core.py' at line 118
 ## 18 <module> file '/usr/bin/nosetests' at line 9
 (Pydb) up
 (/home/yoh/proj/numpy/numpy/lib/npyio.py:212):  close
 212             self.zip.close()
 (Pydb)
 (/home/yoh/proj/numpy/numpy/lib/npyio.py:220):  __del__
 220         self.close()
 (Pydb)
 (/home/yoh/deb/gits/pkg-exppsy/pynn/test/unittests/test_files.py:107):
 test_NumpyBinaryFile
 107     metadata_ = nbf.get_metadata()
 (Pydb) down
 (/home/yoh/proj/numpy/numpy/lib/npyio.py:220):  __del__
 220         self.close()
 (Pydb)
 (/home/yoh/proj/numpy/numpy/lib/npyio.py:212):  close
 212             self.zip.close()
 (Pydb) l
 207             """
 208             Close the file.
 209
 210             """
 211             if self.zip is not None:
 212  ->             self.zip.close()
 213                 self.zip = None
 214             if self.fid is not None:
 215                 self.fid.close()
 216                 self.fid = None
 (Pydb) print self.fid
 <open file 'tmp.npz', mode 'r' at 0x2c2c150>
 *(Pydb) print self.zip.fp == self.fid
 True
 }}}

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/2178#comment:1>
NumPy <http://projects.scipy.org/numpy>
My example project