期刊
INTERNET OF THINGS
卷 16, 期 -, 页码 -出版社
ELSEVIER
DOI: 10.1016/j.iot.2021.100440
关键词
Big data workflows; Internet of Things; Domain-specific languages; Software containers
资金
- EC [101016835, 870130]
- NFR project BigDataMine'' [309691]
- H2020 Societal Challenges Programme [870130] Funding Source: H2020 Societal Challenges Programme
This article presents a Big Data workflow approach based on software container technologies, message-oriented middleware, and a domain-specific language for highly scalable workflow execution and abstract workflow definition. Demonstrations and experiments show the practical applicability of the approach and compare its scalability with that of Argo Workflows, providing a qualitative evaluation of the proposed DSL and overall approach.
Big Data processing, especially with the increasing proliferation of Internet of Things (IoT) technologies and convergence of IoT, edge and cloud computing technologies, involves handling massive and complex data sets on heterogeneous resources and incorporating different tools, frameworks, and processes to help organizations make sense of their data collected from various sources. This set of operations, referred to as Big Data workflows, requires taking advantage of Cloud infrastructures' elasticity for scalability. In this article, we present the design and prototype implementation of a Big Data workflow approach based on the use of software container technologies, message-oriented middleware (MOM), and a domain-specific language (DSL) to enable highly scalable workflow execution and abstract workflow definition. We demonstrate our system in a use case and a set of experiments that show the practical applicability of the proposed approach for the specification and scalable execution of Big Data workflows. Furthermore, we compare our proposed approach's scalability with that of Argo Workflows - one of the most prominent tools in the area of Big Data workflows - and provide a qualitative evaluation of the proposed DSL and overall approach with respect to the existing literature.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据