Edge AI as a Service: Configurable Model Deployment and Delay-Energy Optimization With Result Quality Constraints

The breakthrough of artificial intelligence (AI) techniques has accelerated their applications in a wide range of industries, such as security protection, transportation, agriculture, andmedical care. With the support of edge computing environments, providing latency guaranteed AI as a Service (AIaaS) can accelerate the deployment of data-intensive and computation-intensive AI applications and reduce the investment cost of the customers. However, the deployment architecture and workingmechanism design, and performance optimization problems specific for AIaaS with configurable data quality andmodel complexity have not been studied in existingworks. To address the problem, we propose a configurablemodel deployment architecture (CMDA) for edge AIaaS and present a flexibleworkingmechanismby enabling the joint configuration of data quality ratios (DQRs) andmodel complexity ratios (MCRs) for the AI tasks. Along with commonly used resource allocation operations, themanager can improve the energy and delay performance of AI serviceswith the desired quality of results (QoRs). We develop an energy-delayminimization problemunder the framework ofCMDA and propose a polynomial regression based relaxingmethod to solve the task configuration subproblem. We conduct experiments and simulations on the ImageNet classification and the common objects in context (COCO) object detection tasks using state-of-the-art deep learningmodels. We present the corresponding result quality tables (RQTs) and QoR regressionmodels to illustrate the proposedmethod. The results of single task configuration andmulti-task configuration and resource allocation on ImageNet classification and COCOobject detection tasks demonstrate that the proposedmethod can achieve over 5x HDEC improvement compared with non-optimization schemes, and also show that joint configuration ofDQRandMCR can achieve over 1:2x HDEC improvement compared with the methods that only configureDQRor MCR.

Automated summary

The breakthrough of AI techniques has accelerated their applications in various industries, including security protection, transportation, agriculture, and medical care. With the support of edge computing environments, providing AIaaS with latency guarantee can speed up the deployment of data-intensive and computation-intensive AI applications and reduce customers' investment cost. However, existing studies have not addressed the specific deployment architecture, working mechanism design, and performance optimization problems for AIaaS with configurable data quality and model complexity. To tackle this, we propose a configurable model deployment architecture (CMDA) for edge AIaaS and a flexible working mechanism that allows joint configuration of data quality ratios (DQRs) and model complexity ratios (MCRs) for AI tasks.