[SciPy-user] Re: data smoothing: interpolate.splrep ignores s
pearu at scipy.org
Wed Feb 25 14:45:53 CST 2004
On Wed, 25 Feb 2004, george young wrote:
> [[reposting with "rough" data file appended, sorry]]
> [SciPy-0.2.0_alpha_200.4161, Numeric-23.1, Python 2.3.3, x86 Linux]
I recommend using CVS version of Scipy.
> My goal really is to smooth some noisy measurement data without messing
> up it's *shape*. My first attempt was 1d splines. I did:
> from Numeric import *
> import scipy
> f = file('rough')
> data = array([(float(x),float(y)) for (x,y) in [l.split() for l in f]])
> rep = scipy.interpolate.splrep(data[:,0], data[:,1], s=s)
> smooth_y = scipy.interpolate.splev(data[:,0], rep)
> "s" is supposed to vary the amount of smoothing. For s=0, i get
> my original data back, as expected. But for all other values, including
> the recommended range of (m-sqrt(2.0*m)) to (m+sqrt(2.0*m), I get a
> single, too much smoothed, result. It seems like splrep is not using
> the "s" value to adjust the splines, except to sense that it's not zero.
First note that you have to specify x data as an increasing sequence of
numbers, currently data[:,0] is decreasing.
Second, your data is quite smooth, so you must use very small s values to
see the effect, e.g. try for example s=1e-11 or smaller.
> Splines may not be the right method anyway, since they tend to warp the
> shape of the curve, and I need to get the data's derivatives. Is there
> some way to fashion a low pass filter? It seems like fft should be useful
> here, but I have very little experience with fft's.
FFT is not suitable here as your data is not periodic. But if you need
only derivatives, then use
rep = \
y_der = rep(data[:,0],1) # first derivative
You may need to play with s value a bit and may be also split your data
around x=-0.5 and do the smoothing on these parts separately because the
data behaves differently on those two parts.
More information about the SciPy-user