Analyzing time and power efficiency of machine learning models on edge devices

Edge machine learning (EdgeML) refers to the execution of machine learning (ML) algorithms on devices located close to data sources. The primary goals of EdgeML are to reduce response time and preserve data privacy. Nevertheless, edge devices often face constraints in processing power, memory, and energy, making it challenging to deploy complex ML models, including neural networks (NNs). To address these limitations, significant efforts have focused on improving the time and power efficiency of EdgeML through model optimization and the use of hardware accelerators. Orthogonal to these efforts, this paper introduces EdgeMLProfiler, a novel open-source tool (https://gitlab.kit.edu/uexfc/EdgeMLProfiler), designed to evaluate the time and power consumption of training and inference processes for selected NNs across various hardware architectures and software libraries. Using EdgeMLProfiler, we present–for the first time–a comparative time and power analysis of several NN models, including widely used convolutional neural networks (CNNs) and custom-designed fully-connected neural networks (FNNs). Our analysis reveals distinct efficiency patterns across different models and hardware configurations, providing practical insights for selecting the most time- and power-efficient deployment configurations for ML models on edge devices.