Talking Traces: Audio Reconstruction Power Side-Channel Attack on Neural Network FPGA Accelerators

Machine learning is taking over increasingly many tasks in today's life. A very prevalent application is the use of deep neural networks in chatbots and virtual assistants, which process private texts, images and audio recordings of end users. As the computational requirements for the large neural network models needed are high, the processing is mostly done in the cloud and smaller companies often rely on publicly available pretrained models. This raises security and privacy concerns, as neural network models and the hardware accelerators running them can be targeted by adversaries. In this work, we use remote side-channel analysis to attack a cloud hardware accelerator for a speech recognition application to recover private input audio. The attack uses a generative CNN trained using a deep feature perceptual loss to reconstruct the inputs. For improved results, we add a denoising step using a diffusion model. We show that spectrograms used to represent audio can be recovered from fluctuations in power consumption on an FPGA-based accelerator. These recovered spectrograms often have a high enough quality for a human to recognize the word contained in a recovered audio clip. ... mehrFeeding the spectrograms back to the victim neural network as input results in a recognition accuracy of up to 81.0% at 35 word classes, proving our attack to be effective.