KIT | KIT-Bibliothek | Impressum | Datenschutz

Establishing a Benchmark Dataset for Traceability Link Recovery between Software Architecture Documentation and Models

Fuchß, Dominik ORCID iD icon 1; Corallo, Sophie ORCID iD icon 1,2; Keim, Jan ORCID iD icon 1; Speit, Janek; Koziolek, Anne ORCID iD icon 1
1 Institut für Informationssicherheit und Verlässlichkeit (KASTEL), Karlsruher Institut für Technologie (KIT)
2 Kompetenzzentrum für angewandte Sicherheitstechnologie (KASTEL), Karlsruher Institut für Technologie (KIT)

Abstract (englisch):

In research, evaluation plays a key role to assess the performance of an approach.
When evaluating approaches, there is a wide range of possible types of studies that can be used, each with different properties.
Benchmarks have the benefit that they establish clearly defined standards and baselines.
However, when creating new benchmarks, researchers face various problems regarding the identification of potential data, its mining, as well as the creation of baselines.
As a result, some research domains do not have any benchmarks at all.
This is the case for traceability link recovery between software architecture documentation and software architecture models.
In this paper, we create and describe an open-source benchmark dataset for this research domain.
With this benchmark, we define a baseline with a simple approach based on information retrieval techniques.
This way, we provide other researchers a way to evaluate and compare their approaches.

Volltext §
DOI: 10.5445/IR/1000151962
Veröffentlicht am 27.10.2022
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Informationssicherheit und Verlässlichkeit (KASTEL)
Kompetenzzentrum für angewandte Sicherheitstechnologie (KASTEL)
Publikationstyp Forschungsbericht/Preprint
Publikationsjahr 2022
Sprache Englisch
Identifikator KITopen-ID: 1000151962
HGF-Programm 46.23.01 (POF IV, LK 01) Methods for Engineering Secure Systems
Verlag Karlsruher Institut für Technologie (KIT)
Umfang 8 S.
Projektinformation SofDCar (BMWK, 19S21002K)
Bemerkung zur Veröffentlichung Accepted by 2nd International Workshop on Mining Software Repositories for Software Architecture (MSR4SA’22) - Co-located with 16th European Conference on Software Architecture (ECSA) 2022
Schlagwörter Software Architecture Documentation, Natural Language Processing, Traceability link recovery, Mining Software Repositories
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page