Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm

Purpose To predict the spatial and temporal trajectories of lung tumor during radiotherapy monitored under a longitudinal magnetic resonance imaging (MRI) study via a deep learning algorithm for facilitating adaptive radiotherapy (ART). Methods We monitored 10 lung cancer patients by acquiring weekly MRI-T2w scans over a course of radiotherapy. Under an ART workflow, we developed a predictive neural network (P-net) to predict the spatial distributions of tumors in the coming weeks utilizing images acquired earlier in the course. The three-step P-net consisted of a convolutional neural network to extract relevant features of the tumor and its environment, followed by a recurrence neural network constructed with gated recurrent units to analyze trajectories of tumor evolution in response to radiotherapy, and finally an attention model to weight the importance of weekly observations and produce the predictions. The performance of P-net was measured with Dice and root mean square surface distance (RMSSD) between the algorithm-predicted and experts-contoured tumors under a leave-one-out scheme. Results Tumor shrinkage was 60% +/- 27% (mean +/- standard deviation) by the end of radiotherapy across nine patients. Using images from the first three weeks, P-net predicted tumors on future weeks (4, 5, 6) with a Dice and RMSSD of (0.78 +/- 0.22, 0.69 +/- 0.24, 0.69 +/- 0.26), and (2.1 +/- 1.1 mm, 2.3 +/- 0.8 mm, 2.6 +/- 1.4 mm), respectively. Conclusion The proposed deep learning algorithm can capture and predict spatial and temporal patterns of tumor regression in a longitudinal imaging study. It closely follows the clinical workflow, and could facilitate the decision-making of ART. A prospective study including more patients is warranted.