4.7 Article

An Efficient Organization Method for Large-Scale and Long Time-Series Remote Sensing Data in a Cloud Computing Environment

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JSTARS.2021.3110900

关键词

Remote sensing; Spatiotemporal phenomena; Organizations; Metadata; Distributed databases; Cloud computing; Indexing; Cloud computing; data management; data organization; remote sensing; time series

资金

  1. National Natural Science Foundation of China [41801363, 41925007]

向作者/读者索取更多资源

This study proposes an efficient data organization method for improving retrieval and access efficiencies of large-scale and long time-series remote-sensing data in a cloud-computing environment. It constructs an asymmetrical index model and prepartitioning mechanism to address low retrieval efficiency, and divides remote-sensing images into tiles with consistent hash operations to enhance access efficiency.
Historical earth observation (EO) data have played an important role in long-term scientific and environmental monitoring. The effective organization of large-scale and long-term remote-sensing data to achieve efficient retrieval and access has become one of the important issues. However, inherent big data characteristics, such as a large scale, and asymmetrical temporal and spatial distributions, have caused problems with the efficiency of data retrieval and access. Therefore, this study proposes an efficient data organization method for use in a cloud-computing environment that has two aims. First, it addresses the problem of low retrieval efficiency. An asymmetrical index model for the image metadata is constructed that is based on a unified spatio-temporal grid coding; a prepartitioning mechanism under the HBase architecture is established to realize the uniform storage of the metadata with an asymmetrical spatiotemporal distribution and to avoid retrieval efficiency bottlenecks caused by a load imbalance. Second, it addresses low access efficiency. By dividing the remote-sensing image into tiles, a unified spatio-temporal code is established for each tile, and a consistent hash operation is performed; tiles with similar hash values are stored in the same or adjacent Hadoop Distributed File System nodes. In this way, tiles with temporal or spatial correlations can be gathered and stored, and lots of disk seeks can be avoided during retrieval, thereby greatly improving the data access efficiency. Comparative experiments showed that the data organization method can effectively improve the retrieval and access efficiencies of large-scale and long time-series remote-sensing data in a cloud-computing environment.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据