Xel: A cloud-agnostic data platform for the design-driven building of high-availability data science services

This paper presents Xel, a cloud-agnostic data platform for the design-driven building of highavailability data science services as a support tool for data-driven decision-making. We designed and implemented Xel based on four main components: (a) a high level and driven-design framework for end-users to select analytic and machine learning tools from a service mesh and coupling them into the form of processing pipelines; (b) a new recursive ETL processing model to automatically convert the pipeline designs into infrastructure-agnostic software structures, which are deployed on multiple infrastructures; (c) an orchestration model for transparently managing the data delivery throughout each stage of the processing pipelines used in data science systems; and (d) a data decentralized model to transparently mask service unavailability such as cloud outages and unavailability of either applications or data. Real users created, by means of Xel, data science services such as deep learning analysis of scientific publications, clustering of movie reviews, and a cancer exploratory study. These services were evaluated as case studies that revealed the efficacy of this platform design for enabling end-users to create multiple types of data science pipelines without programming nor making configurations, and automatically masking unavailability of cloud resources and data. This platform is currently used to create a national cancer observatory and big data systems for fusing suicide, mental health, drug consumption and macroeconomic datasets to find spatiotemporal patterns. (c) 2023 Elsevier B.V. All rights reserved.

Automated summary

This paper presents Xel, a cloud-agnostic data platform designed to support the building of high-availability data science services for data-driven decision-making. Xel includes a framework for end-users to select analytic and machine learning tools, a recursive ETL processing model, an orchestration model for managing data delivery, and a data decentralized model to mask service unavailability. Real users have successfully created various data science services using Xel, and the platform has been proven effective in enabling users to create pipelines without programming and automatically masking unavailability of cloud resources and data.