KIT | KIT-Bibliothek | Impressum | Datenschutz

A Benchmark Data Set to Evaluate the Illumination Robustness of Image Processing Algorithms for Object Segmentation and Classification

Khan, Arif ul Maula 1; Mikut, Ralf ORCID iD icon 1; Reischl, Markus ORCID iD icon 1
1 Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung (IOSB)


Developers of image processing routines rely on benchmark data sets to give qualitative comparisons of new image analysis algorithms and pipelines. Such data sets need to include artifacts in order to occlude and distort the required information to be extracted from an image. Robustness, the quality of an algorithm related to the amount of distortion is often important. However, using available benchmark data sets an evaluation of illumination robustness is difficult or even not possible due to missing ground truth data about object margins and classes and missing information about the distortion. We present a new framework for robustness evaluation. The key aspect is an image benchmark containing 9 object classes and the required ground truth for segmentation and classification. Varying levels of shading and background noise are integrated to distort the data set. To quantify the illumination robustness, we provide measures for image quality, segmentation and classification success and robustness. We set a high value on giving users easy access to the new benchmark, therefore, all routines are provided within a software package, but can as well easily be replaced to emphasize other aspects.

Volltext §
DOI: 10.5445/IR/1000049767
DOI: 10.1371/journal.pone.0131098
Zitationen: 6
Web of Science
Zitationen: 5
Zitationen: 6
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Angewandte Informatik (IAI)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2015
Sprache Englisch
Identifikator ISSN: 1932-6203
KITopen-ID: 1000049767
HGF-Programm 47.01.02 (POF III, LK 01) Biol.Netzwerke u.Synth.Regulat. IAI
Erschienen in PLoS one
Verlag Public Library of Science (PLoS)
Band 10
Heft 7
Seiten Art.Nr. e0131098
Bemerkung zur Veröffentlichung Gefördert durch den KIT-Publikationsfonds
Nachgewiesen in Scopus
Web of Science
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page