Spline-Based Density Estimation Minimizing Fisher Information

The construction of a continuous probability density function (pdf) that fits a set of samples is a frequently occurring task in statistics. This is an inherently underdetermined problem, that can only be solved by making some assumptions about the samples or the distribution to be estimated. This paper proposes a density estimation method based on the premise that each sample represents the same amount of probability mass of the underlying density. The estimated pdf is parameterized as the square of a polynomial spline, which makes further processing of the estimated density very efficient. This pdf is inherently nonnegative, ensuring a monotone cumulative distribution function, which makes it easy to generate samples from it through inverse transform sampling. Furthermore, it is cheap to evaluate and easy to integrate, making moment calculations fast. To find the coefficients of the polynomials that make up the spline, an optimization problem is derived. The Fisher information is used as a regularizer in this problem to select the solution that contains the least amount of information. The method is shown to work on samples from a variety of different one-dimensional probability distributions.