Elicitation and Classification of Security Requirements for EVerest

Incomplete and unverified requirements can lead to misunderstandings and misconceptions. Specifically in security, violated requirements can be indicators for potential vulnerabilities. To avoid vulnerabilities, security reuqirements are linked to their implementations. For further verification, security analyses can be used to check whether a software design fulfills its required properties. Therefore, specific attributes of requirements have to be identified and linked to the design. In this thesis, security requirements for the open-source software EVerest are elicited. EVerest provides a full stack environment for electric vehicle charging stations. For the elicitation, a two-step approach is used. First, a questionnaire is developed that elicits coarse-grained requirements of the security categories confidentiality, integrity, availability, and authentication. Afterwards, four EVerest software developers are interviewed to refine the coarse-grained requirements to 93 design-level security requirements. Prompt engineering and fine-tuning are used to classify design elements and extract their respective mentions from the retrieved requirements. ... mehrGPT’s quality for the design element classification and extraction of their mentions is determined in three evaluation scenarios both for individual elements and per requirement. First, it is examined how well GPT classifies the elements in the requirements. Then, GPT’s ability to extract their mentions in the requirements is determined. Finally, it is examined how well GPT extracts the mention of the design element in the requirement in case the design element is correctly classified. The results indicate that regarding multi-class design element classification in requirements works well in prompt engineering and fine-tuning (F1-score: 0.67-0.73). However, regarding the extraction of design elements, fine-tuning (F1-score: 0.7) surpasses prompt engineering (F1-score: 0.52). If both tasks are combined, fine-tuning (F1-score: 0.87) also outperforms prompt engineering (F1-score: 0.61). However, in prompt engineering, GPT seems to be inappropriate to classify all design elements or extract all mentions per requirement, while the results in fine-tuning for the design element classification (F1-score: 0.18-0.2) and the mention extraction (F1-score: 0.15-0.18) are acceptable for a multi-label classification.