[SciPy-user] Re: data smoothing: interpolate.splrep ignores s parameter

Pearu Peterson pearu at scipy.org
Wed Feb 25 14:45:53 CST 2004



On Wed, 25 Feb 2004, george young wrote:

> [[reposting with "rough" data file appended, sorry]]
> [SciPy-0.2.0_alpha_200.4161, Numeric-23.1, Python 2.3.3, x86 Linux]

I recommend using CVS version of Scipy.

> My goal really is to smooth some noisy measurement data without messing
> up it's *shape*.  My first attempt was 1d splines.  I did:
> 
> from Numeric import *
> import scipy
> f = file('rough')
> data = array([(float(x),float(y)) for (x,y) in [l.split() for l in f]])
> rep = scipy.interpolate.splrep(data[:,0], data[:,1], s=s)
> smooth_y = scipy.interpolate.splev(data[:,0], rep)
> 
> "s" is supposed to vary the amount of smoothing.  For s=0, i get
> my original data back, as expected.  But for all other values, including
> the recommended range of (m-sqrt(2.0*m)) to (m+sqrt(2.0*m), I get a
> single, too much smoothed, result.  It seems like splrep is not using
> the "s" value to adjust the splines, except to sense that it's not zero.

First note that you have to specify x data as an increasing sequence of 
numbers, currently data[:,0] is decreasing.

Second, your data is quite smooth, so you must use very small s values to 
see the effect, e.g. try for example s=1e-11 or smaller.

> Splines may not be the right method anyway, since they tend to warp the
> shape of the curve, and I need to get the data's derivatives.  Is there
> some way to fashion a low pass filter?  It seems like fft should be useful
> here, but I have very little experience with fft's.

FFT is not suitable here as your data is not periodic. But if you need 
only derivatives, then use

rep = \
 interpolate.fitpack2.UnivariateSpline(data[::-1,0],data[::-1,1],s=1e-13)

y_der = rep(data[:,0],1) # first derivative

You may need to play with s value a bit and may be also split your data 
around x=-0.5 and do the smoothing on these parts separately because the 
data behaves differently on those two parts.

HTH,
Pearu



More information about the SciPy-user mailing list