Development and Validation of the Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale (ROADS)

Importance A new outcome measure for overall disability level with improved responsiveness is needed for amyotrophic lateral sclerosis (ALS) clinical trials. Objective To describe the creation and development of a new self-reported ALS disability scale with improved item targeting and psychometric properties that used a mathematically rigorous Rasch methodology. Design, Setting, and Participants A preliminary ALS disability questionnaire with 119 questions was created based on literature review, clinical judgement of an expert panel, and patient input. Patients with ALS were recruited from January 2017 to June 2019 from the Emory University and Atlanta VA Medical Center ALS clinics, both in Atlanta, Georgia, during regularly scheduled clinic appointments to complete the draft questionnaire and standard ALS outcome measures. All consecutive patients seen at the Emory University and Atlanta VA Medical Center ALS clinics during the recruitment period with a diagnosis of ALS who were able to provide informed consent were invited to participate in the study. Rasch analyses were performed, and items were systematically removed based on missing data, model fit, disordered thresholds, item bias, and clinical judgment. A total of 509 patients with ALS were seen at the 2 sites during the recruitment period, and 264 patients provided informed consent. Interventions Participants completed the draft Rasch questionnaire and the revised Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS-R). Main Outcomes and Measures Rasch analyses and standard scale metrics were performed to create the new scale, and Rasch analyses were performed on the ALSFRS-R for comparison. Results Overall, 243 participants with ALS completed the draft questionnaire, and 230 participants were included for Rasch analyses. The mean (SD) age for study participants was 61.9 (11.1) years, 146 (60.1%) were men, and site of onset was 23.0% bulbar (n = 56), 36.2% upper extremity (n = 88), and 39.5% lower extremity (n = 96). A 28-question Rasch-Built Overall ALS Disability Scale (ROADS) was constructed with each item scored 0, 1, or 2. The ROADS fulfilled Rasch model requirements, demonstrated improved item targeting compared with the ALSFRS-R, and had test-retest reliability of 0.97. Individual question fit statistics demonstrated infit values from 0.68 to 1.37 and outfit values from 0.66 to 1.43. The difference between the empirical variance explained by the measures and the modeled variance was 0.1%. The ALSFRS-R violated Rasch model expectations and demonstrated disordered thresholds for 9 of 12 questions; 13 of 48 answer choices on the ALSFRS-R were never the most probable answer choice for any overall disability level. Conclusions and Relevance In this study, the 28-question, self-reported ROADS, which is linearly weighted, had improved item targeting compared with the ALSFRS-R, had high test-retest reliability, and was validated. ROADS may serve as a valuable and easily accessible outcome measure for use in ALS trials and in the clinic with improved responsiveness compared with the ALSFRS-R. Question By combining clinical expertise with mathematically rigorous Rasch methodology, can a new self-reported amyotrophic lateral sclerosis disability scale be developed with improved reliability and responsiveness compared with the ordinal scale in current use? Findings In this study of 243 individuals, the 28-question Rasch-Built Overall Amyotrophic Lateral Sclerosis Disability Scale was created and had improved item targeting and high test-retest reliability. Meaning This new Rasch-Built disability scale may serve as an easily accessible outcome measure that could improve the efficiency of amyotrophic lateral sclerosis clinical trials and serve as an informative tool in the clinic setting. This study describes the development and validation of a new self-reported amyotrophic lateral sclerosis disability scale.