Hardware-Centric Exploration of the Discrete Design Space in Transformer-LSTM Models for Wind Speed Prediction on Memory-Constrained Devices

Wind is one of the most important resources in the renewable energy basket. However, questions regarding wind as a sustainable solution, especially when it faces challenges such as upfront cost, visual impact, noise pollution and bird collisions. These challenges arise from commercial windmills whereas for domestic small-scale windmills these challenges are limited. On the other hand, accurate wind speed prediction (WSP) is crucial for optimizing power management in renewable energy systems. Existing research focuses on proposing model architectures and optimizing hyper-parameters to improve model performance. This approach often results in larger models, which are hosted on cloud servers. Such models face challenges, including bandwidth utilization leading to data delays, increased costs, security risks, concerns about data privacy and the necessity for continuous internet connectivity. Such resources are not available for domestic windmills. To overcome these obstacles, this work proposes a transformer model integrated with Long Short-Term Memory (LSTM) units, optimized for memory-constrained devices (MCD). A contribution of this research is the development of a novel cost function that balances the reduction of mean square error with the constraints of model size. This approach enables model deployment on low-power devices, avoiding the challenges of cloud-based deployment. The model, with its tuned hyper-parameters, outperforms recent methodologies in terms of mean squared error, mean absolute error, model size and R-squared scores across three different datasets. This advancement leads the way for more dynamic and secure on-device wind speed prediction (WSP) applications, representing a step forward in renewable energy management.