[SciPy-user] A first proposal for dataset organization
Wed Sep 19 09:05:48 CDT 2007
2007/9/19, Anne Archibald <email@example.com>:
> On 18/09/2007, David Huard <firstname.lastname@example.org> wrote:
> > For large data sets, I'm not sure I understand what you're meaning. Do
> > intend to include netcdf or HDF5 files and provide an interface to
> > those data sets so users don't have to bother about the underlying
> engine ?
> > Do we really want to distribute a package weighting > 1GB ?
> One of the points of this project, as I understand it, is to make it
> convenient for people to get and use real datasets. In particular, one
> possibility is to not include the data in this package, but instead
> only a script to download it from (say) the HEASARC. Thus big datasets
> are not outrageous, and more to the point, we need to be able to deal
> with them whatever form they are in natively.
My understanding was rather :
" ... to make it convenient for people to get and use real datasets for use
in SciPy and NumPy examples, documentation and tutorials. " This limits the
scope of the dataset package, at least for starters. If some tutorial deals
with larger than memory issues, then using a specialized binary format makes
sense. However, I think that pretty basic datasets can illustrate the use of
most SciPy and NumPy functions.
> SciPy-user mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SciPy-user