[NumPy-Tickets] [NumPy] #1660: ~2**32 byte tofile()/fromfile() limit in 64-bit Windows

[NumPy Trac]

#1660: ~2**32 byte tofile()/fromfile() limit in 64-bit Windows
------------------------------------------------------+---------------------
 Reporter:  mspacek                                   |       Owner:  somebody    
     Type:  defect                                    |      Status:  needs_review
 Priority:  normal                                    |   Milestone:  1.5.1       
Component:  numpy.core                                |     Version:  1.5.0       
 Keywords:  tofile fromfile save load 64-bit windows  |  
------------------------------------------------------+---------------------
Changes (by mspacek):

  * keywords:  tofile fromfile save 64-bit windows => tofile fromfile save
               load 64-bit windows
  * status:  new => needs_review


Comment:

 I've tried out Christoph's patch, as provided in his latest binary at
 http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy, and it all seems to work
 now! Here are the tests I ran. They do take a fair while to run. 6GB seems
 a reasonable trade-off between speed and ensuring there's no size limit:

 {{{
 SIXGB = 2**32 + 2**31

 np.zeros(SIXGB, np.int8).tofile('test.npy')
 a = np.fromfile('test.npy', dtype=np.int8)
 assert len(a) == SIXGB

 np.save('test.npy', np.ones(SIXGB, np.int8))
 a = np.load('test.npy')
 assert len(a) == SIXGB
 assert (a == 1).all() # this doubles memory usage temporarily
 assert a.sum(dtype=np.int64) == SIXGB
 }}}

 I've never written unit tests before. I'll see what I can learn and try
 and submit something to the ticket.

-- 
Ticket URL: <http://projects.scipy.org/numpy/ticket/1660#comment:9>
NumPy <http://projects.scipy.org/numpy>
My example project