KIT | KIT-Bibliothek | Impressum | Datenschutz

Expected Information Maximization: Using the I-Projection for Mixture Density Estimation

Becker, Philipp; Arenz, Oleg; Neumann, Gerhard

Abstract (englisch):
Modelling highly multi-modal data is a challenging problem in machine learning.
Most algorithms are based on maximizing the likelihood, which corresponds
to the M(oment)-projection of the data distribution to the model distribution.
The M-projection forces the model to average over modes it cannot represent.
In contrast, the I(nformation)-projection ignores such modes in the data and concentrates on the modes the model can represent.
Such behavior is appealing whenever we deal with highly multi-modal data where modelling single modes correctly is more important than covering all the modes.
Despite this advantage, the I-projection is rarely used in practice due to the lack of algorithms that can efficiently optimize it based on data.
In this work, we present a new algorithm called Expected Information Maximization (EIM) for computing the I-projection solely based on samples for general latent variable models, where we focus on Gaussian mixtures models and Gaussian mixtures of experts.
Our approach applies a variational upper bound to the I-projection objective which decomposes the original objective into single objectives for each mixture component as well as for the coefficients, allowing an efficient optimization. ... mehr

Open Access Logo


Verlagsausgabe §
DOI: 10.5445/IR/1000118434
Veröffentlicht am 21.04.2020
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2020
Sprache Englisch
Identifikator KITopen-ID: 1000118434
Erschienen in Eigth International Conference on Learning Representations (ICLR 2020)
Bemerkung zur Veröffentlichung ICLR2020 as a Fully Virtual Conference: April 25 - April 30 2020
Externe Relationen Siehe auch
Schlagwörter density estimation, information projection, mixture models, generative learning, multimodal modeling
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page