Detecting Nonequivalence in Neural Networks Through In-Distribution Counterexample Generation

Neural networks (NNs) have made profound achievements in various safety-critical applications such as healthcare, medical devices, and automotive. These NN models are usually trained using cloud systems; however, due to latency, privacy, and bandwidth concerns, inference is performed on edge devices. Consequently, the model size is often reduced through pruning and quantization to map the cloud-trained models to edge artificial intelligence hardware. To ensure that the reduced models maintain the integrity of the original, larger models, detecting inequivalences is crucial. In this letter, we focus on inequivalence detection by identifying cases where the behavior of the reduced model diverges from the original model. This is achieved by formulating an optimization problem to maximize the difference between the two models. In contrast to the related work, our proposed approach is agnostic to the choice of activation function and can be applied to networks utilizing a wide variety of nonlinearities. Furthermore, it considers only counterexamples that are in range of the original data, the so-called In Distribution, as only in these regions, the model can be considered properly specified. ... mehrThe experimental results showed that the found counterexamples were able to differentiate models for various NN architectures and datasets.