Selective Sampling: A Complexity-Aware Sampling Strategy for Combinatorial Scenario-based Testing

The SOTIF-standard (ISO 21448) requires scenario-based testing to verify and validate advanced driver assistance systems and automated driving systems. Feature modeling and sampling have shown promising results for generating scenarios considered test cases for scenario-based testing. Sampling strategies commonly applied for generating scenarios pursue coverage-criteria such as t-wise feature interaction, but ignore the number of selected features in a valid configuration. In the context of scenario generation, the number of features in a configuration correlates to the complexity of a scenario; thus, considering the number of features is relevant for "sufficiently" covering the scenario space and generating SOTIF-compliant scenarios. In this paper, we propose a complexity-aware coverage criterion embedded in selective sampling as a complexity-aware sampling strategy. Selective sampling approximates and replicates the distribution of the number of selected features of valid configurations. We apply selective sampling to generate scenarios for testing two advanced driver assistance systems. Our experiments demonstrate that selective sampling has the potential to improve scenario generation.