Logical big data integration and near real-time data analytics

In the context of decision-making, there is a growing demand for near real-time data that traditional solutions, like data warehousing based on long-running ETL processes, cannot fully meet. On the other hand, existing logical data integration solutions are challenging because users must focus on data location and distribution details rather than on data analytics and decision-making. EasyBDI is an open-source system that provides logical integration of data and high-level business-oriented abstractions. It uses schema matching, integration, and mapping techniques, to automatically identify partitioned data and propose a global schema. Users can then specify star schemas based on global entities and submit analytical queries to retrieve data from distributed data sources without knowing the organization and other technical details of the underlying systems. This work presents the algorithms and methods for global schema creation and query execution. Experimental results show that the overhead imposed by logical integration layers is relatively small compared to the execution times of distributed queries.

Automated summary

In the field of decision-making, there is a growing need for real-time data that traditional solutions cannot meet. Existing logical data integration solutions also present challenges as they require users to focus on data details rather than analytics. EasyBDI is an open-source system that offers logical data integration and business-oriented abstractions. It automatically identifies partitioned data and proposes a global schema, allowing users to retrieve data from distributed sources without technical knowledge. Experimental results show minimal overhead compared to distributed query execution times.