4.4 Article

Logical big data integration and near real-time data analytics

Journal

DATA & KNOWLEDGE ENGINEERING
Volume 146, Issue -, Pages -

Publisher

ELSEVIER
DOI: 10.1016/j.datak.2023.102185

Keywords

Big data integration; Distributed databases; Near real-time OLAP

Ask authors/readers for more resources

In the field of decision-making, there is a growing need for real-time data that traditional solutions cannot meet. Existing logical data integration solutions also present challenges as they require users to focus on data details rather than analytics. EasyBDI is an open-source system that offers logical data integration and business-oriented abstractions. It automatically identifies partitioned data and proposes a global schema, allowing users to retrieve data from distributed sources without technical knowledge. Experimental results show minimal overhead compared to distributed query execution times.
In the context of decision-making, there is a growing demand for near real-time data that traditional solutions, like data warehousing based on long-running ETL processes, cannot fully meet. On the other hand, existing logical data integration solutions are challenging because users must focus on data location and distribution details rather than on data analytics and decision-making. EasyBDI is an open-source system that provides logical integration of data and high-level business-oriented abstractions. It uses schema matching, integration, and mapping techniques, to automatically identify partitioned data and propose a global schema. Users can then specify star schemas based on global entities and submit analytical queries to retrieve data from distributed data sources without knowing the organization and other technical details of the underlying systems. This work presents the algorithms and methods for global schema creation and query execution. Experimental results show that the overhead imposed by logical integration layers is relatively small compared to the execution times of distributed queries.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.4
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available