[SciPy-User] R: Re: R: Re: R: Re: R: Re: Epanechnikov kernel

francescoboccacci@libe... francescoboccacci@libe...
Sat Jan 19 12:39:02 CST 2013


Great. Thanks Patrick.I'll let you know.
Francesco


----Messaggio originale----

Da: patrickmarshwx@gmail.com

Data: 19/01/2013 18.47

A: "francescoboccacci@libero.it"<francescoboccacci@libero.it>, "SciPy Users List"<scipy-user@scipy.org>

Ogg: Re: [SciPy-User] R: Re: R: Re: R: Re: Epanechnikov kernel



I should also add that you can approximate an Epanechnikov kernel with a Gaussian kernel. See: http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-11-00200.1


The take away line is: 
"Using the results of Marron and Nolan (1988), it can be shown that, when comparing Epanechnikov and Gaussian kernels, the  Epanechnikov kernel must be 2.2138 times larger than the Gaussian bandwidth to achieve a similar response function."


So, you can take the bandwidth you'd like to use with the Epanechnikov kernel, and divide it by 2.2138 and plug the result into the Gaussian kernel. It's not exact, but the response is similar.



Patrick
---
Patrick Marsh
Ph.D. Candidate / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies


National Severe Storms Laboratory
http://www.patricktmarsh.com



On Sat, Jan 19, 2013 at 11:46 AM, Patrick Marsh <patrickmarshwx@gmail.com> wrote:


I apologize if this is a duplicate...I used the wrong email initially and wasn't sure if it would go through the listserv....








I've previously coded up a Cython version of the Epanechnikov kernel.  You can find the function here:



https://gist.github.com/4573808


It's certainly not optimized. It was a quick hack for use with rare (spatial) meteorological events. As the grid density increases, the performance decreases significantly. At this point, your best bet would be to create a grid that has the weights of the Epanechnikov kernel, and do a FFT convolve between the two grids. A pseudocode example (that I believe should work) is shown below...





============================================
import numpy as npimport scipy as sp
import epanechnikov (from the gist linked to above)



data_to_kde = ... # Your 2D array

# Create a grid with a value of 1 at the midpoint
raw_epan_grid = np.zeros((51, 51), dtype=np.float64)raw_epan_gird[25, 25] = 1



# Convert this binary grid into the weights of the Epanechnikov kernel


bandwidth = 10dx = 1
epan_kernel = epanechnikov(raw_epan_grid, bandwidth, dx)



# Use FFTCONVOLVE to do the smoothing in Fourier space


data_smoothed = sp.signal.fftconvolve(data_to_kde, epan_kernel, mode='same')============================================



This is slower than the function linked above for sparse grids, but faster for dense grids. (The runtime of fftconvolve is dependent upon the size of your arrays,  not the density.)





Hope this helps
Patrick---
Patrick Marsh
Ph.D. Candidate / Liaison to the HWT
School of Meteorology / University of Oklahoma
Cooperative Institute for Mesoscale Meteorological Studies
National Severe Storms Laboratory



http://www.patricktmarsh.com



On Sat, Jan 19, 2013 at 9:18 AM, francescoboccacci@libero.it <francescoboccacci@libero.it> wrote:



Thanks Josef, i will investigate on it.

I'm using scipy version '0.9.0' so i need to update it.

If i have some problems i will ask you again :).

Thanks for your time



Francesco



>----Messaggio originale----

>Da: josef.pktd@gmail.com

>Data: 19/01/2013 16.06

>A: "francescoboccacci@libero.it"<francescoboccacci@libero.it>, "SciPy Users



List"<scipy-user@scipy.org>

>Ogg: Re: [SciPy-User] R: Re: R: Re: Epanechnikov kernel

>

>On Sat, Jan 19, 2013 at 9:57 AM, francescoboccacci@libero.it

><francescoboccacci@libero.it> wrote:

>> Hi,

>> i would like to use a Epanechnikov kernel because i would  like replicate

an R

>> function that use Epanechnikov kernel.

>> Reading in depth a documentation below documentation:

>>

>>

>> http://rgm3.lab.nig.ac.jp/RGM/r_function?p=adehabitatHR&amp;f=kernelUD

>>

>> i found that i can use normal kernel (i think guaussion kernel).

>> Below i write a pieces of my code:

>>

>>

>>                   xmin = min(xPoints)

>>                   xmax = max(xPoints)

>>                   ymin = min(yPoints)

>>                   ymax = max(yPoints)

>>                   X,Y = np.mgrid[xmin:xmax:40j, ymin:ymax:40j]

>>                   positions = np.vstack([X.ravel(), Y.ravel()])

>>                   values = np.vstack([xPoints,yPoints])

>>                   # scipy.stats.kde.gaussian_kde --

>>                   # Representation of a kernel-density estimate using

Gaussian

>> kernels.

>>                   kernel = stats.kde.gaussian_kde(values)

>>

>>                   Z = np.reshape(kernel(positions).T, X.T.shape)

>>

>> If i understood in right way the missing part that i have to implement is

the

>> smoothing paramter h:

>>

>> h = Sigma*n^(-1/6)

>>

>> where

>>

>> Sigma = 0.5*(sd(x)+sd(y))

>>

>>

>> My new question is:

>>

>> How can set smooting parameter in stats.kde.gaussian_kde function? is it

>> possible?

>

>In a recent scipy (since 0.10 IIRC) you can directly set the bandwidth

>without subclassing

>

>http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.gaussian_kde.

html#scipy.stats.gaussian_kde

>http://docs.scipy.org/doc/scipy/reference/tutorial/stats.html#kernel-density-

estimation

>

>Josef

>

>>

>> Thanks

>>

>> Francesco

>>

>>

>>>----Messaggio originale----

>>>Da: jsseabold@gmail.com

>>>Data: 19/01/2013 15.21

>>>A: "francescoboccacci@libero.it"<francescoboccacci@libero.it>, "SciPy Users



>> List"<scipy-user@scipy.org>

>>>Ogg: Re: [SciPy-User] R: Re: Epanechnikov kernel

>>>

>>>On Sat, Jan 19, 2013 at 8:48 AM, francescoboccacci@libero.it

>>><francescoboccacci@libero.it> wrote:

>>>> Hi,

>>>> is there a possibility to multivariate  KDE using Epanechnikov kernel? my

>>>> variables are X Y (point position)

>>>>

>>>

>>>As Josef mentioned there is no way for the user to choose the kernel

>>>at present. The functionality is there, but it needs to be hooked in

>>>with a suitable API. I didn't keep up with these discussions, so I

>>>don't know the current status. If it's something you're interested in

>>>trying to help with, I'm sure people would be appreciative and you can

>>>ping the statsmodels mailing list.

>>>

>>>Practically though, the reason this hasn't been done yet is that the

>>>choice of the kernel is not all that important. Bandwidth selection is

>>>the most important variable and other kernels perform similarly given

>>>a good bandwidth. Is there any particular reason you want Epanechnikov

>>>kernel in particular?

>>>

>>>Skipper

>>>

>>>> Thanks

>>>>

>>>> Francesco

>>>>

>>>>>----Messaggio originale----

>>>>>Da: jsseabold@gmail.com

>>>>>Data: 19/01/2013 14.32

>>>>>A: "SciPy Users List"<scipy-user@scipy.org>

>>>>>Ogg: Re: [SciPy-User] Epanechnikov kernel

>>>>>

>>>>>On Sat, Jan 19, 2013 at 7:49 AM,  <josef.pktd@gmail.com> wrote:

>>>>>> On Sat, Jan 19, 2013 at 6:34 AM, francescoboccacci@libero.it

>>>>>> <francescoboccacci@libero.it> wrote:

>>>>>>> Hi all,

>>>>>>>

>>>>>>> I have a question for you. Is it possible in scipy using a

Epanechnikov

>>>>>>> kernel function?

>>>>>>>

>>>>>>> I checked on scipy documentation but i found that the only way to

>>>> calculate

>>>>>>> kernel-density estimate is possible only with using Gaussian kernels?

>>>>>>>

>>>>>>> Is it true?

>>>>>>

>>>>>> Yes, kde in scipy.stats only has gaussian_kde

>>>>>>

>>>>>> Also in statsmodels currently only gaussian is supported for

>>>>>> continuous data

>>>>>> http://statsmodels.sourceforge.net/devel/nonparametric.html

>>>>>> (It was removed because in the references only the bandwidth selection

>>>>>> made much difference in the estimation, but not the shape of the

>>>>>> kernel. Other kernels for continuous variables will come back

>>>>>> eventually.

>>>>>

>>>>>If you're interested in univariate KDE, then we do have the Epanechnikov

>>>> kernel.

>>>>>

>>>>>http://statsmodels.sourceforge.net/devel/generated/statsmodels.

>> nonparametric.

>>>> kde.KDEUnivariate.fit.html#statsmodels.nonparametric.kde.KDEUnivariate.

fit

>>>>>

>>>>>Skipper

>>>>>_______________________________________________

>>>>>SciPy-User mailing list

>>>>>SciPy-User@scipy.org

>>>>>http://mail.scipy.org/mailman/listinfo/scipy-user

>>>>>

>>>>

>>>>

>>>> _______________________________________________

>>>> SciPy-User mailing list

>>>> SciPy-User@scipy.org

>>>> http://mail.scipy.org/mailman/listinfo/scipy-user

>>>

>>

>>

>> _______________________________________________

>> SciPy-User mailing list

>> SciPy-User@scipy.org

>> http://mail.scipy.org/mailman/listinfo/scipy-user

>





_______________________________________________

SciPy-User mailing list

SciPy-User@scipy.org

http://mail.scipy.org/mailman/listinfo/scipy-user









 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20130119/50fec3b2/attachment-0001.html 


More information about the SciPy-User mailing list