A Sustainable Data Extraction Documentation Process for Literature Studies

Background
Open Science advocates for transparency of research outcomes; however, software engineering (SE) research often lacks adequate availability of essential artifacts. This deficiency hinders reproducibility, sustainability, and the broader adoption of Open Science practices. Evidence-Based Software Engineering (EBSE) emphasizes the systematic use of empirical evidence to evaluate SE practices. Systematic literature studies form a cornerstone of EBSE but face challenges in persistency, particularly for data extraction results in a multi-reviewer context, due to limited machine-readable formats and inefficiencies in reviewer collaboration.
Objective.
This paper introduces a practical guideline and documentation process for conducting systematic literature studies with a categorical classification objective (e.g., scoping studies). The approach extends existing methodologies by integrating tool support in terms of version control and machine-readable formats, thereby reinforcing Open Science principles.
Method.
We performed a feature analysis, following the framework of Marshall et al., to evaluate the suitability of selected tools for supporting documentation and coordination in multi-reviewer data extraction scenarios. ... mehr
Results.
Our proposal for a tool-supported data extraction achieves a high score in the feature analysis, indicating suitability and enhancement of common practices. Our guideline improves the documentation of data extraction results in literature studies and addresses key challenges related to traceability, consistency, and reviewer coordination.
Conclusion.
Research artifacts are critical for the sustainability of SE research and the advancement of Open Science. Our proposed guideline provides a structured, tool-supported approach that enhances transparency, reproducibility, and efficiency in EBSE studies.