Genetic Algorithm-Based Online-Partitioning BranchyNet for Accelerating Edge Inference

In order to effectively apply BranchyNet, a DNN with multiple early-exit branches, in edge intelligent applications, one way is to divide and distribute the inference task of a BranchyNet into a group of robots, drones, vehicles, and other intelligent edge devices. Unlike most existing works trying to select a particular branch to partition and deploy, this paper proposes a genetic algorithm (GA)-based online partitioning approach that splits the whole BranchyNet with all its branches. For this purpose, it establishes a new calculation approach based on the weighted average for estimating total execution time of a given BranchyNet and a two-layer chromosome GA by distinguishing partitioning and deployment during the evolution in GA. The experiment results show that the proposed algorithm can not only result in shorter execution time and lower device-average energy cost but also needs less time to obtain an optimal deployment plan. Such short running time enables the proposed algorithm to generate an optimal deployment plan online, which dynamically meets the actual requirements in deploying an intelligent application in the edge.

Automated summary

To effectively apply BranchyNet in edge intelligent applications, this paper proposes a genetic algorithm-based online partitioning approach to split the whole network into multiple devices. The approach achieves shorter execution time, lower energy cost, and faster optimization of the deployment plan. The results demonstrate the algorithm's capability to dynamically meet the requirements of deploying intelligent applications at the edge.