Journal
2017 46TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING (ICPP)
Volume -, Issue -, Pages 231-240Publisher
IEEE COMPUTER SOC
DOI: 10.1109/ICPP.2017.32
Keywords
erasure code; redundancy scheme; distributed storage system
Categories
Funding
- National High Technology Research and Development Program (863 Program) of China [2013AA013203]
- NSFC [61502190, 61232004]
- National Basic Research 973 Program of China [2011CB302301]
Ask authors/readers for more resources
Modern distributed storage systems often store redundant data in multiple replications or erasure coding according to their access frequencies. Multiple replications scheme is well-performance for hot data while erasure coding scheme is storage-efficient for warm and cold data. When hot data turn cold, an encoding procedure starts to do the conversion. However, due to sequential striping, current conversion methods do not perform well for different data layouts, and cause risky blocks and expensive network consumption. In this paper, we propose Sice, a new encoder which deploys non-sequential striping. It constructs non-sequential stripes according to the data layout, performs conversion quickly with low overheads and ends to no reduction of system reliability. The results of both simulation and evaluation show that Sice gains almost the same good performance for different data layouts and has a great scalability. Sice helps HDFS-RAID reduce network consumption by about 65% and reduce influence on concurrent I/O-intensive applications by about 60%.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available