[Numpy-discussion] numarray memmap

[Todd Miller]

On Wed, 2003-12-03 at 13:47, Sebastian Haase wrote:
> Hi,
> I would like to use memmap array more often - here are some comments:
> (Please look for lines with $$$$)
> 
> >>> from numarray import memmap as mm
> >>> m = mm.open("C:/mm1", mode='w+', len=0)
> >>> n = m.insert(0,0)
> >>> a = na.array(buffer=n)
> X:\PrWin\wxPython\lib\PyCrust\PyCrustApp.py:1: DeprecationWarning: The
> 'buffer' keyword is deprecated.  Use 'sequence' instead.
> Exception exceptions.AttributeError: "Memmap instance has no attribute
> '_mode'" in Traceback (most recent call last):
>   File "<input>", line 1, in ?
>   File "X:\PrWin\numarray\numarraycore.py", line 279, in array
>     type = getTypeObject(sequence, type, typecode)
>   File "X:\PrWin\numarray\numarraycore.py", line 223, in getTypeObject
>     return Py2NumType[ _maxtype(sequence) ]
> KeyError
> 
> $$$$ Is this a mistake on my part or should the error message be "better"
> ???


How's this?:

Traceback (most recent call last):
  File "sabby1.py", line 5, in ?
    a = na.array(buffer=n)
  File
"/home/jmiller/lib/python2.3/site-packages/numarray/numarraycore.py",
line 282, in array
    type = getTypeObject(sequence, type, typecode)
  File
"/home/jmiller/lib/python2.3/site-packages/numarray/numarraycore.py",
line 226, in getTypeObject
    raise TypeError("Can't determine a reasonable type from sequence")
TypeError: Can't determine a reasonable type from sequence

> 
> >>> a = na.array(buffer=n, type=na.UInt16, shape=(0,0,0))
> >>> a.shape
> (0, 0, 0)
> >>> m
> <Memmap on file 'C:/mm1' with mode='w+', length=0, 1 slices>
> >>> a.resize((100,100,100))
> >>> a.shape
> (100, 100, 100)
> >>> m.sync()
> >>> m.flush()
> Traceback (most recent call last):
>   File "<input>", line 1, in ?
>   File "X:\PrWin\numarray\memmap.py", line 501, in flush
>     self._consolidate(filename)
>   File "X:\PrWin\numarray\memmap.py", line 469, in _consolidate
>     f.write(self._buffer(m, mlen))
>   File "X:\PrWin\numarray\memmap.py", line 317, in _buffer
>     raise RuntimeError("Memmap no longer valid.  (closed?)")
> RuntimeError: Memmap no longer valid.  (closed?)
> >>> m
> <Memmap on file 'C:/mm1' with mode='w+', length=0, 1 slices>
> 
> $$$$  Is this a bug ?

Evidently.  I logged it on SF.

Switching to a length 1 map seems to "fix" the problem so I think you've
discovered an interesting edge case bug in _consolidate().  I'm still
looking for the fix.

> 
> My idea is to "prepare" the memmap file before I actually know the array
> dimensions that should go into it.  Is that a good idea ?

In theory, yes.  In reality... apparently not.

> 
> Thanks,
> Sebastian Haase

Regards,
Todd

> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21030
(410) 338 - 4576