A Multicenter Longitudinal MRI Study Assessing LeMan-PV Software Accuracy in the Detection of White Matter Lesions in Multiple Sclerosis Patients

BackgroundDetecting new and enlarged lesions in multiple sclerosis (MS) patients is needed to determine their disease activity. LeMan-PV is a software embedded in the scanner reconstruction system of one vendor, which automatically assesses new and enlarged white matter lesions (NELs) in the follow-up of MS patients; however, multicenter validation studies are lacking. PurposeTo assess the accuracy of LeMan-PV for the longitudinal detection NEL white-matter MS lesions in a multicenter clinical setting. Study TypeRetrospective, longitudinal. SubjectsA total of 206 patients with a definitive MS diagnosis and at least two follow-up MRI studies from five centers participating in the Swiss Multiple Sclerosis Cohort study. Mean age at first follow-up = 45.2years (range: 36.9-52.8years); 70 males. Field Strength/SequenceFluid attenuated inversion recovery (FLAIR) and T1-weighted magnetization prepared rapid gradient echo (T1-MPRAGE) sequences at 1.5 T and 3 T. AssessmentThe study included 313 MRI pairs of datasets. Data were analyzed with LeMan-PV and compared with a manual reference standard provided by a neuroradiologist. A second rater (neurologist) performed the same analysis in a subset of MRI pairs to evaluate the rating-accuracy. The Sensitivity (Se), Specificity (Sp), Accuracy (Acc), F1-score, lesion-wise False-Positive-Rate (aFPR), and other measures were used to assess LeMan-PV performance for the detection of NEL at 1.5 T and 3 T. The performance was also evaluated in the subgroup of 123 MRI pairs at 3 T. Statistical TestsIntraclass correlation coefficient (ICC) and Cohen's kappa (CK) were used to evaluate the agreement between readers. ResultsThe interreader agreement was high for detecting new lesions (ICC = 0.97, Pvalue<10(-20), CK = 0.82, P value = 0) and good (ICC = 0.75, P value<10(-12), CK = 0.68, P value = 0) for detecting enlarged lesions. Across all centers, scanner field strengths (1.5 T, 3 T), and for NEL, LeMan-PV achieved: Acc = 61%, Se = 65%, Sp = 60%, F1-score = 0.44, aFPR = 1.31. When both follow-ups were acquired at 3 T, LeMan-PV accuracy was higher (Acc = 66%, Se = 66%, Sp = 66%, F1-score = 0.28, aFPR = 3.03). Data ConclusionIn this multicenter study using clinical data settings acquired at 1.5 T and 3 T, and variations in MRI protocols, LeMan-PV showed similar sensitivity in detecting NEL with respect to other recent 3 T multicentric studies based on neural networks. While LeMan-PV performance is not optimal, its main advantage is that it provides automated clinical decision support integrated into the radiological-routine flow. Evidence Level4 Technical EfficacyStage 2

Automated summary

This study evaluated the accuracy of LeMan-PV software for detecting new and enlarged white matter lesions in multiple sclerosis patients. The results showed that LeMan-PV had similar sensitivity in detecting new lesions compared to other recent studies using neural networks. Although its performance is not optimal, the main advantage of LeMan-PV is that it provides automated clinical decision support integrated into the routine radiological workflow.