Accelerating Gossip-Based Deep Learning in Heterogeneous Edge Computing Platforms

With the exponential growth of data created at the network edge, decentralized and Gossip-based training of deep learning (DL) models on edge computing (EC) gains tremendous research momentum, owing to its capability to learn from resource-strenuous edge nodes with limited network connectivity. Today's edge devices are extremely heterogeneous, e.g., hardware and software stacks, and result in high performance variation of training time and inducing extra delay to synchronize and converge. The large body of prior art accelerates DL, being data or model parallelization, via a centralized server, e.g., parameter server scheme, which may easily turn into the system bottleneck or single point of failure. In this artice, we propose EdgeGossip, a framework specifically designed to accelerate the training process of decentralized and Gossip-based DL training for heterogeneous EC platforms. EdgeGossip features on: (i) low performance variation among multiple EC platforms during iterative training, and (ii) accuracy-aware training to fastly obtain best possible model accuracy. We implement EdgeGossip based on popular Gossip algorithms and demonstrate its effectiveness using real-world DL workloads, i.e., considerably reducing model training time by an average of 2.70 times while only incurring accuracy losses of 0.78 percent.

Automated summary

Research shows that with the exponential growth of data generated by edge computing, the decentralized and Gossip-based training of deep learning models is gaining momentum. The EdgeGossip framework is designed to reduce the performance variation among heterogeneous edge platforms during training and achieve best possible model accuracy quickly. Implementing EdgeGossip based on popular Gossip algorithms has demonstrated an average reduction of model training time by 2.70 times with only 0.78% accuracy loss.