A Comparative Study of Pruning Methods in Transformer-Based Time Series Forecasting

The high parameter count and corresponding computational demand of transformer-based models for time-series forecasting poses a challenge to training and real-world deployment. Pruning is an established approach to reduce compute, but its effects have not yet been studied on Transformer-based models for time series forecasting. To close this gap, we provide a comparative benchmark study by evaluating unstructured and structured pruning on various state-of-the-art multivariate time series models. Our results show that fine-tuning some pruned models is necessary. Furthermore, we demonstrate that even with corresponding hardware and software support, structured pruning is unable to provide significant time savings.