Analysis and mathematical modeling of big data processing

Big data processing is an urgent and unresolved challenge that originates from the intensive development of information technology. The recent techniques lose their effectiveness rapidly as the volumes of data increase. In this article, we will put down our vision of the basic approaches and models related to problem solving, based on processing large data volumes. This article introduces a two-stage decomposition of a problem, related to assessing management options. The first stage of our original approach implies a semantic analysis of textual information; the second stage is built around finding association rules in a database, processing them via mathematical statistics methods, and converting data and objectives to a vector. We suggest processing the collected news events by a semantic model, which describes their key features and interconnections between them in a specified subject area. The classification-based association rules allow assessing the likelihood of a particular event using a set chain of events. This approach can be applied through the analysis of online news in a specified market segment.

Automated summary

Big data processing, stemming from intensive development of information technology, presents an urgent challenge. This article proposes a two-stage problem decomposition approach involving semantic analysis and association rule mining to tackle large data volumes. By processing news events using a semantic model and classification-based association rules, the likelihood of specific events in a market segment can be assessed effectively.