Mapping Paddy Rice Using Weakly Supervised Long Short-Term Memory Network with Time Series Sentinel Optical and SAR Images

Rice is one of the most important staple food sources worldwide. Effective and cheap monitoring of rice planting areas is demanded by many developing countries. This study proposed a weakly supervised paddy rice mapping approach based on long short-term memory (LSTM) network and dynamic time warping (DTW) distance. First, standard temporal synthetic aperture radar (SAR) backscatter profiles for each land cover type were constructed on the basis of a small number of field samples. Weak samples were then labeled on the basis of their DTW distances to the standard temporal profiles. A time series feature set was then created that combined multi-spectral Sentinel-2 bands and Sentinel-1 SAR vertical received (VV) band. With different combinations of training and testing datasets, we trained a specifically designed LSTM classifier and validated the performance of weakly supervised learning. Experiments showed that weakly supervised learning outperformed supervised learning in paddy rice identification when field samples were insufficient. With only 10% of field samples, weakly supervised learning achieved better results in producer's accuracy (0.981 to 0.904) and user's accuracy (0.961 to 0.917) for paddy rice. Training with 50% of field samples also presented improvement with weakly supervised learning, although not as prominent. Finally, a paddy rice map was generated with the weakly supervised approach trained on field samples and DTW-labeled samples. The proposed data labeling approach based on DTW distance can reduce field sampling cost since it requires fewer field samples. Meanwhile, validation results indicated that the proposed LSTM classifier is suitable for paddy rice mapping where variance exists in planting and harvesting schedules.