[Numpy-discussion] Huge arrays

David Cournapeau david@ar.media.kyoto-u.ac...
Thu Sep 10 02:06:21 CDT 2009


Kim Hansen wrote:
>
>     On 9-Sep-09, at 4:48 AM, Francesc Alted wrote:
>
>     > Yes, this later is supported in PyTables as long as the underlying
>     > filesystem
>     > supports files > 2 GB, which is very usual in modern operating
>     > systems.
>
>     I think the OP said he was on Win32, in which case it should be noted:
>     FAT32 has its upper file size limit at 4GB (minus one byte), so
>     storing both your arrays as one file on a FAT32 partition is a no-no.
>
>     David
>
>  
> Strange, I work on Win32 systems, and there I have no problems storing
> data files up to 600 GB (have not tried larger) in size stored on
> RAID0 disk systems of 2x1TB, I can also open them and seek in them
> using Python.

It is a FAT32 limitation, not a windows limitation. NTFS should handle
large files without much trouble, and I believe the vast majority of
windows installations (>= windows xp) use NTFS and not FAT32. I
certainly have not seen a windows installed on FAT32 for a very long time.

cheers,

David


More information about the NumPy-Discussion mailing list