Readability of English, German, and Russian Disease-Related Wikipedia Pages: Automated Computational Analysis

Background: Wikipedia is a popular encyclopedia for health- and disease-related information in which patients seek advice and guidance on the web. Yet, Wikipedia articles can be unsuitable as patient education materials, as investigated in previous studies that analyzed specific diseases or medical topics with a comparatively small sample size. Currently, no data are available on the average readability levels of all disease-related Wikipedia pages for the different localizations of this particular encyclopedia. Objective: This study aimed to analyze disease-related Wikipedia pages written in English, German, and Russian using well-established readability metrics for each language. Methods: Wikipedia database snapshots and Wikidata metadata were chosen as resources for data collection. Disease-related articles were retrieved separately for English, German, and Russian starting with the main concept of Human Diseases and Disorders (German: Krankheit; Russian:). In the case of existence, the corresponding International Classification of Diseases, Tenth Revision (ICD-10), codes were retrieved for each article. Next, the raw texts were extracted and readability metrics were computed. Results: The number of articles included in this study for English, German, and Russian Wikipedia was n=6127, n=6024, and n=3314, respectively. Most disease-related articles had a Flesch Reading Ease (FRE) score <50.00, signaling difficult or very difficult educational material (English: 5937/6125, 96.93%; German: 6004/6022, 99.7%; Russian: 2647/3313, 79.9%). In total, 70% (7/10) of the analyzed articles could be assigned an ICD-10 code with certainty (English: 4235/6127, 69.12%; German: 4625/6024, 76.78%; Russian: 2316/3314, 69.89%). For articles with ICD-10 codes, the mean FRE scores were 28.69 (SD 11.00), 20.33 (SD 9.98), and 38.54 (SD 13.51) for English, German, and Russian, respectively. A total of 9 English ICD-10 chapters (11 German and 10 Russian) showed significant differences: chapter F (FRE 23.88, SD 9.95; P<.001), chapter E (FRE 25.14, SD 9.88; P<.001), chapter H (FRE 30.04, SD 10.57; P=.049), chapter I (FRE 30.05, SD 9.07; P=.04), chapter M (FRE 31.17, 11.94; P<.001), chapter T (FRE 32.06, SD 10.51; P=.001), chapter A (FRE 32.63, SD 9.25; P<.001), chapter B (FRE 33.24, SD 9.07; P<.001), and chapter S (FRE 39.02, SD 8.22; P<.001). Conclusions: Disease-related English, German, and Russian Wikipedia articles cannot be recommended as patient education materials because a major fraction is difficult or very difficult to read. The authors of Wikipedia pages should carefully revise existing text materials for readers with a specific interest in a disease or its associated symptoms. Special attention should be given to articles on mental, behavioral, and neurodevelopmental disorders (ICD-10 chapter F) because these articles were most difficult to read in comparison with other ICD-10 chapters. Wikipedia readers should be supported by editors providing a short and easy-to-read summary for each article.

Automated summary

This study analyzed disease-related Wikipedia pages written in English, German, and Russian using readability metrics. The results showed that most disease-related articles on Wikipedia are difficult or very difficult to read, suggesting they are not suitable as patient education materials.