KIT | KIT-Bibliothek | Impressum | Datenschutz

Computing accurate probabilistic estimates of one‐d entropy from equiprobable random samples

Gupta, Hoshin V.; Ehsani, Mohammad Reza; Roy, Tirthankar; Sans-fuentes, Maria A.; Ehret, Uwe 1; Behrangi, Ali
1 Institut für Wasser und Gewässerentwicklung (IWG), Karlsruher Institut für Technologie (KIT)

Abstract:

We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) and Kernel Density (KD) methods. In contrast to BC, which uses equal-width bins with varying probability mass, the QS method uses estimates of the quantiles that divide the support of the data generating probability density function (pdf) into equal-probability-mass intervals. And, whereas BC and KD each require optimal tuning of a hyper-parameter whose value varies with sample size and shape of the pdf, QS only requires specification of the number of quantiles to be used. Results indicate, for the class of distributions tested, that the optimal number of quantiles is a fixed fraction of the sample size (empirically determined to be ~0.25–0.35), and that this value is relatively insensitive to distributional form or sample size. This provides a clear advantage over BC and KD since hyper-parameter tuning is not required. Further, unlike KD, there is no need to select an appropriate kernel-type, and so QS is applicable to pdfs of arbitrary shape, including those with discontinuous slope and/or magnitude. ... mehr


Verlagsausgabe §
DOI: 10.5445/IR/1000135252
Veröffentlicht am 13.07.2021
Originalveröffentlichung
DOI: 10.3390/e23060740
Scopus
Zitationen: 2
Web of Science
Zitationen: 3
Dimensions
Zitationen: 4
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Wasser und Gewässerentwicklung (IWG)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2021
Sprache Englisch
Identifikator ISSN: 1099-4300
KITopen-ID: 1000135252
Erschienen in Entropy
Verlag MDPI
Band 23
Heft 6
Seiten 740
Nachgewiesen in Dimensions
Web of Science
Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page