Toward Decentralized and Collaborative Deep Learning Inference for Intelligent IoT Devices

Deep learning technologies are empowering IoT devices with an increasing number of intelligent services. However, the contradiction between resource-constrained IoT devices and intensive computing makes it common to transfer data to the cloud center for executing all DNN inference, or dynamically allocate DNN computations between IoT devices and the cloud center. Existing approaches perform a strong dependence on the cloud center, and require the support of a reliable and stable network. Thus, it may directly cause unreliable or even unavailable service in extreme or unstable environments. We propose DeColla, a decentralized and collaborative deep learning inference system for IoT devices, which completely migrates DNN computations from the cloud center to the IoT device side, relying on the collaborative mechanism to accelerate the DNN inference that is difficult for an individual IoT device to accomplish. DeColla uses a parallel acceleration strategy via a DRL-based adaptive allocation for collaborative inference, which aims to improve inference efficiency and robustness. To illustrate the advantages and robustness of DeColla, we built a testbed and employ DeColla to evaluate MobileNet DNN network trained on the ImageNet dataset, and also recognize the object for a mobile web AR application and conduct extensive experiments to analyze the latency, resource usage, and robustness against existing methods. Numerical results show that DeColla outperforms other methods in terms of latency and resource usage, which can especially reduce at least 2.5 times latency than the hierarchical inference method when the collaboration is interrupted abnormally.

Automated summary

This paper proposes a decentralized and collaborative deep learning inference system, DeColla, which migrates DNN computations from the cloud center to IoT devices to improve efficiency and robustness. Experimental results show that DeColla outperforms other methods in terms of latency and resource usage.