[Numpy-discussion] checksum on numpy float array

Brennan Williams brennan.williams@visualreservoir....
Thu Dec 4 17:43:58 CST 2008


josef.pktd@gmail.com wrote:
> On Thu, Dec 4, 2008 at 6:17 PM, Brennan Williams
> <brennan.williams@visualreservoir.com> wrote:
>   
>> My app reads in one or more float arrays from a binary file.
>>
>> Sometimes due to network timeouts etc the array is not read correctly.
>>
>> What would be the best way of checking the validity of the data?
>>
>> Would some sort of checksum approach be a good idea?
>> Would that work with an array of floating point values?
>> Or are checksums more for int,byte,string type data?
>>
>>     
>
> If you want to verify the file itself, then python provides several
> more or less secure checksums, my experience was that zlib.crc32 was
> pretty fast on moderate file sizes. crc32 is common inside archive
> files and for binary newsgroups. If you have large files transported
> over the network, e.g. GB size, I would work with par2 repair files,
> which verifies and repairs at the same time.
>
>   
The file has multiple arrays stored in it.

So I want to have some sort of validity check on just the array that I'm 
reading.

I will need to add a check on the file as well as of course network 
problems could affect writing to the file
as well as reading from the file.


> Josef
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>   



More information about the Numpy-discussion mailing list