期刊
JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY
卷 35, 期 1, 页码 47-60出版社
SCIENCE PRESS
DOI: 10.1007/s11390-020-9798-5
关键词
high performance computing; I; O interference; parallel file system; performance optimization; resource misallocation
资金
- National Key Research and Development Program of China [2016YFB1000504]
- Natural Science Foundation of China [61433008, 61373145, 61572280]
- China Postdoctoral Science Foundation [2018M630162]
It is hard for applications to make full utilization of the peak bandwidth of the storage system in highperformance computers because of I/O interferences, storage resource misallocations and complex long I/O paths. We performed several studies to bridge this gap in the Sunway storage system, which serves the supercomputer Sunway TaihuLight. To locate these issues and connections between them, an end-to-end performance monitoring and diagnosis tool was developed to understand I/O behaviors of applications and the system. With the help of the tool, we were about to find out the root causes of such performance barriers at the I/O forwarding layer and the parallel file system layer. An application-aware I/O forwarding allocation framework was used to address the I/O interferences and resource misallocations at the I/O forwarding layer. A performance-aware data placement mechanism was proposed to mitigate the impact of I/O interferences and performance variations of storage devices in the PFS. Together, applications obtained much better I/O performance. During the process, we also proposed a lightweight storage stack to shorten the I/O path of applications with -N I/O pattern. This paper summarizes these studies and presents the lessons learned from the process.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据