[Numpy-discussion] Huge arrays
Thu Sep 10 02:06:21 CDT 2009
Kim Hansen wrote:
> On 9-Sep-09, at 4:48 AM, Francesc Alted wrote:
> > Yes, this later is supported in PyTables as long as the underlying
> > filesystem
> > supports files > 2 GB, which is very usual in modern operating
> > systems.
> I think the OP said he was on Win32, in which case it should be noted:
> FAT32 has its upper file size limit at 4GB (minus one byte), so
> storing both your arrays as one file on a FAT32 partition is a no-no.
> Strange, I work on Win32 systems, and there I have no problems storing
> data files up to 600 GB (have not tried larger) in size stored on
> RAID0 disk systems of 2x1TB, I can also open them and seek in them
> using Python.
It is a FAT32 limitation, not a windows limitation. NTFS should handle
large files without much trouble, and I believe the vast majority of
windows installations (>= windows xp) use NTFS and not FAT32. I
certainly have not seen a windows installed on FAT32 for a very long time.
More information about the NumPy-Discussion