Accurate Prediction of Workloads and Resources With Multi-Head Attention and Hybrid LSTM for Cloud Data Centers

Currently, cloud computing service providers face big challenges in predicting large-scale workload and resource usage time series. Due to the difficulty in capturing nonlinear features, traditional forecasting methods usually fail to achieve high prediction performance for resource usage and workload sequences. Besides, there is much noise in original time series of resources and workloads. If these time series are not de-noised by smoothing algorithms, the prediction results can fail to meet the providers' requirements. To do so, this work proposes a hybrid prediction model named VAMBiG that integrates Variational mode decomposition, an Adaptive Savitzky-Golay (SG) filter, a Multi-head attention mechanism, Bidirectional and Grid versions of Long and Short Term Memory (LSTM) networks. VAMBiG adopts a signal decomposition method named variational mode decomposition to decompose complex and non-linear original time series into low-frequency intrinsic mode functions. Then, it adopts an adaptive SG filter as a data pre-processing tool to eliminate noise and extreme points in such functions. Afterwards, it adopts bidirectional and grid LSTM networks to capture bidirectional features and dimension ones, respectively. Finally, it adopts a multi-head attention mechanism to explore importance of different data dimensions. VAMBiG aims to predict resource usage and workloads in highly variable traces in clouds. Extensive experimental results demonstrate that it achieves higher-accuracy prediction than several advanced prediction approaches with datasets from Google and Alibaba cluster traces.

Automated summary

This study proposes a hybrid prediction model called VAMBiG that integrates various techniques to address the challenges faced by cloud computing service providers in predicting large-scale workload and resource usage time series. The model utilizes signal decomposition, data pre-processing, bidirectional and grid LSTM networks, and a multi-head attention mechanism to achieve higher prediction accuracy. Experimental results demonstrate its superiority over several advanced prediction approaches.