期刊
IEEE TRANSACTIONS ON COMPUTERS
卷 72, 期 5, 页码 1314-1328出版社
IEEE COMPUTER SOC
DOI: 10.1109/TC.2022.3214113
关键词
Computer architecture; Field programmable gate arrays; Dynamic scheduling; Optimization; Hardware; Bandwidth; Parallel processing; Multi-tenancy; deep neural network; multi-core; accelerator; FPGA
This paper proposes the H3M framework to optimize the architecture, scheduling, and mapping for INFaaS on cloud FPGA. H3M outperforms other accelerators in terms of EDP reduction on the ASIC platform. On the Xilinx U200 and U280 FPGA platforms, H3M significantly reduces EDP compared to Herald.
Deep Neural Network (DNN) INFerence-as-a-Service (INFaaS) is the dominating workload in current data centers, for which FPGAs become promising hardware platforms because of their high flexibility and energy efficiency. The dynamic and multi-tenancy nature of INFaaS requires careful design in three aspects: multi-tenant architecture, multi-DNN scheduling, and multi-core mapping. These three factors are critical to the system latency and energy efficiency but are also challenging to optimize since they are tightly coupled and correlated. This paper proposes H3M, an automatic Design Space Exploration (DSE) framework to jointly optimize the architecture, scheduling, and mapping for serving INFaaS on cloud FPGAs. H3M explores: (1) the architecture design space with Heterogeneous spatial Multi-tenant sub-accelerators, (2) layer-wise scheduling for Heterogeneous Multi-DNN workloads, and (3) single-layer mapping to the Homogeneous Multi-core architecture. H3M beats state-of-the-art multi-tenant DNN accelerators, Planaria and Herald, by up to 7.5x and 3.6x in Energy-Delay-Product (EDP) reduction on the ASIC platform. On the Xilinx U200 and U280 FPGA platforms, H3M offers 2.1-5.7x and 1.8-9.0x EDP reduction over Herald.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据