期刊
FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE
卷 51, 期 -, 页码 120-131出版社
ELSEVIER SCIENCE BV
DOI: 10.1016/j.future.2014.11.006
关键词
Distributed storage system; Swift storage system; Workload balancing; Resource management
The demand for big data storage and processing has become a challenge in today's industry. To meet the challenge, there is an increasing number of enterprises adopting distributed storage systems. Frequently, in these systems, storage nodes intensively holding hotspot data could become system bottlenecks while storage nodes without hotspot data might result in low utilization of computing resource. This stems from the fact that almost all the typical distributed storage systems only provide data-amount-oriented balancing mechanisms without considering the different access load of data. To eliminate the system bottlenecks and optimize the resource utilization, there is a demand for such distributed storage systems to employ a workload balancing and adaptive resource management framework. In this paper, we propose a framework of workload balancing and resource management for Swift, a widely used and typical distributed storage system on cloud. In this framework, we design workload monitoring and analysis algorithms for discovering overloaded and underloaded nodes in the cluster. To balance the workload among those nodes, Split, Merge and Pair Algorithms are implemented to regulate physical machines while Resource Reallocate Algorithm is designed to regulate virtual machines on cloud. In addition, by leveraging the mature architecture of distributed storage systems, the framework resides in the hosts and operates through API interception. To demonstrate its effectiveness, we conduct experiments to evaluate it. And the experimental results show the framework can achieve its goals. (C) 2014 Elsevier B.V. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据