Kernel density estimation of 100 normally distributed random numbers using different smoothing bandwidths.

The Parzen window is also used in signal processing as a lag window, in such procedures as the Blackman-Tukey procedure.

A histogram can be thought of as a collection of point samples from a kernel density estimate for which the kernel is a uniform box the width of the histogram bin.

Definition

If x₁, x₂, ..., x_N ~ ƒ is an independent and identically-distributed random variables sample of a random variable, then the kernel density approximation of its probability density function is

\widehat{f}_h(x)=\frac{1}{Nh}\sum_{i=1}^N K\left(\frac{x-x_i}{h}\right)

where K is some kernel and h is the bandwidth (smoothing parameter). Quite often K is taken to be a standard Gaussian function with mean zero and variance 1:

K(x) = {1 \over \sqrt{2\pi} }\,e^{-\frac{1}{2}x^2}.

Intuition

Although less smooth density estimators such as the histogram density estimator can be made to be asymptotically consistent, others are often either discontinuous or converge at slower rates than the kernel density estimator. Rather than grouping observations together in bins, the kernel density estimator can be thought to place small "bumps" at each observation, determined by the kernel function. The estimator consists of a "sum of bumps" and is clearly smoother as a result (see below image).

Six Gaussians (red) and their sum (blue). The Parzen window density estimate f(x) is obtained by dividing this sum by 6, the number of Gaussians. The variance of the Gaussians was set to 0.5. Note that where the points are denser the density estimate will have higher values.

Properties

Let \scriptstyle R(f,\hat f(x)) be the L² risk function for ƒ. Under weak assumptions on ƒ and K,

R(f,\hat f(x)) \approx \frac{1}{4}\sigma_k^4h^4\int(f''(x))^2\,dx + \frac{\int K^2(x)\,dx}{nh}\text{ where }\sigma_K^2 = \int x^2K(x)\,dx.

By minimizing the theoretical risk function, it can be shown that the optimal bandwidth is

h^* = \frac{c_1^{-2/5}c_2^{1/5}c_3^{-1/5}}{n^{1/5}}

where

c_1 = \int x^2K(x)\,dx

c_2 = \int K(x)^2\,dx

c_3 = \int (f''(x))^2\,dx.

When the optimal choice of bandwidth is chosen, the risk function is \scriptstyle R(f, \hat f(x))\, \approx\, \tfrac{c_4}{n^{4/5}} for some constant c₄ > 0. It can be shown that, under weak assumptions, there cannot exist a non-parametric estimator that converges at a faster rate than the kernel estimator. Note that the n^−4/5 rate is slower than the typical n⁻¹ convergence rate of parametric methods.

Statistical implementation

• In Matlab, kernel density estimation is implemented through the ksdensity function.
• In Stata, it is implemented through kdensity; for example histogram x, kdensity.
• In R, it is implemented through the density function and the kde2d function.
• In SAS, proc kde can be used to estimate univariate and bivariate kernel densities.
• In SciPy, scipy.stats.gaussian_kde can be used to perform gaussian kernel density estimation in arbitrary dimensions, including bandwidth estimation.

See also

• Kernel (statistics)
• Kernel (mathematics)
• Kernel smoothing
• Mean-shift
• Scale space (The surface in x, bandwidth, Density space for all bandwidths is essentially a scale space representation of the data.)

References

• Parzen E. (1962). On estimation of a probability density function and mode, Ann. Math. Stat. 33, pp. 1065-1076.
• Duda, R. and Hart, P. (1973). Pattern Classification and Scene Analysis. John Wiley & Sons. ISBN 0-471-22361-1.
• Wasserman, L. (2005). All of Statistics: A Concise Course in Statistical Inference, Springer Texts in Statistics.

External links

• Introduction to kernel density estimation
• Free Matlab m-file for one and two dimensional kernel density estimation
• Free Online Software (Calculator) computes the Kernel Density Estimation for any data series according to the following Kernels: Gaussian, Epanechnikov, Rectangular, Triangular, Biweight, Cosine, and Optcosine.
• FIGTree is a fast library that can be used to compute Kernel Density Estimates using a Gaussian Kernel. MATLAB and C/C++ interfaces available.
• Good summary of methods to choose bandwidth Bandwidth Selection in Kernel Density Estimation: A Review.

de:Kerndichteschätzer fr:Estimateur_par_noyau ja:????????