KIT | KIT-Bibliothek | Impressum | Datenschutz

Evaluating the Overlap of User Preferences with Search Engine Performances using UCISE

Abt, Raphael; Haigis, Jonathan; Stappen, Christina


Background: The assessment of information systems such as Search Engines (SEs) from the users’ perspective needs to assure that it does not introduce a bias towards the systems’ technical implementation. Instead, the socio-technical nature of SEs and a users’ contextual evaluation based on previous experiences should be at the center of attention.
Objective: Therefore, we introduce a methodological approach for a User Centered Information System Evaluation (UCISE) and apply it to a SE evaluation. This approach allows for comparing selected SEs based on a derived set of user-based evaluation criteria.
Method: Our methodology consists of four steps, including a criteria synthesis, a criteria importance assessment by a large-scale survey and an expert assessment of selected SE performances regarding the user-based criteria. In the final step, we visualize the fit between the users’ criteria preferences and the assessed SE performances.
Results: The first step of our methodological approach enabled the synthesis of a set of ten criteria for the SE differentiation from user-perspective. Of these ten criteria, only the three criteria ‘search result quality’, ‘ease of use’ and ‘privacy protection’ emerged as being perceived as ‘extremely important’ or ‘important’ by the user. ... mehr

Verlagsausgabe §
DOI: 10.5445/IR/1000138902
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB)
Publikationstyp Buchaufsatz
Publikationsmonat/-jahr 10.2021
Sprache Englisch
Identifikator KITopen-ID: 1000139021
Erschienen in cii Student Papers - 2021. Ed.: A. Sunyaev
Verlag Karlsruher Institut für Technologie (KIT)
Seiten 121-142
Schlagwörter user perspective, user-centered, search engine, information system, evaluation, assessment, Google, DuckDuckGo
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page