[SciPy-User] [newbie] how to compare two datasets
Thøger Emil Juul Thorsen
Tue Jul 13 08:37:04 CDT 2010
That sounds like a set of NumPy arrays is what you need.
You can simply import your dataset to an array and perform row- and
First I would do an interpolation of the real data, though I'd probably
use a cubic spline, but linear is fine too. The spline function will
operate on a numpy array and return the mathematical object, *not* a new
array. This spline can then be evaluated in the depths for which you
have your model data.
An example of how it could be done would be:
modeldata = numpy.genfromtxt('modeled.data')
realdata = numpy.genfromtxt('real.data')
# Now say depth is the first column, and value is second:
tck = scipy.interpolate.splrep(realdata[:, 0], realdata[:, 1])
iplrealdata = scipy.interpolate.splev(modeldata[:, 0], tck)
#You will now have an interpolated value of the real data for every
#depth of the model data - done with a cubic spline.
#Linear interpolation would be done by, instead of doing splev, doing:
func = scipy.interpolate.interp1d(realdata[:, 0], realdata[:, 1])
iplrealdata = func(modeldata[:, 0])
On Tue, 2010-07-13 at 07:00 +0000, ben h wrote:
> The datasets are borehole data - so they have borehole name, depth,
> and a value.
> Each borehole has two datasets - one real, one modelled.
> I want to compare the values between them for each depth in modelled
> dataset (lower resolution / fewer samples).
> If there is no matching depth in real dataset I want to linearly
> interpolate between nearest values.
> Comparison to be quite simple at first, difference between values, and
> stats for entire set of differences.
> example data (depth, value):
> 0 15.5
> -10 17.0
> -20 18.5
> -30 20.0
> 0 16.5
> -1 16.6
> -2 16.6
> -655 55.3
> Not having used python much, i don't know best data structure
> (dictionary? sequence? list?), or if there are helpful things in SciPy
> to help this come together (stats, methods for comparing datasets like
> these, linear interp methods?).
> Looking for inspiration and pointers!
> Find it at CarPoint.com.au New, Used, Demo, Dealer or Private?
> SciPy-User mailing list
More information about the SciPy-User