Journal
JOURNAL OF SUPERCOMPUTING
Volume 74, Issue 2, Pages 615-636Publisher
SPRINGER
DOI: 10.1007/s11227-017-2151-2
Keywords
Elastic scheduling; Data stream graph; Streaming application; High-velocity stream; Big data computing
Categories
Funding
- National Natural Science Foundation of China [61602428]
- Fundamental Research Funds for the Central Universities [2652015338]
- Melbourne-Chindia Cloud Computing (MC3) Research Network
Ask authors/readers for more resources
Online scheduling plays a key role for big data streaming applications in a big data stream computing environment, as the arrival rate of high-velocity continuous data stream might fluctuate over time. In this paper, an elastic online scheduling framework for big data streaming applications (E-Stream) is proposed, exhibiting the following features. (1) Profile mathematical relationships between system response time, multiple application fairness, and online features of high-velocity continuous stream. (2) Scale out or scale in a data stream graph by quantifying computation and communication cost, and the vertex semantics for arrival rate of data stream, and adjust the degree of parallelism of vertices in the graph. Subgraph is further constructed to minimize data dependencies among the subgraphs. (3) Elastically schedule a graph by a priority-based earliest finish time first online scheduling strategy, and schedule multiple graphs by a max-min fairness strategy. (4) Evaluate the low system response time and acceptable applications fairness objectives in a real-world big data stream computing environment. Experimental results conclusively demonstrate that the proposed E-Stream provides better system response time and applications fairness compared to the existing Storm framework.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available