An analysis of the quality of cartilage repair studies

Background: Most lesions of articular cartilage do not heal spontaneously and may lead to secondary osteoarthritis. It is not known whether the optimistic reports on the short and long-term results of several different cartilage repair techniques are based on sound methodological quality. Methods: We performed a literature search in MEDLINE, CINAHL, the Cochrane Central Register, and EMBASE and included studies in which the primary aim of the investigation was to report the outcome after cartilage repair in the knee with use of microfracture, autologous osteochondral transplantation, autologous periosteal transplantation, or autologous chondrocyte implantation. We scored the quality of the studies using a modified Coleman Methodology Score with ten criteria, which results in a final score between 0 and 100. Studies were also assessed with use of the level-of-evidence rating used in the American Volume of The Journal of Bone and Joint Surgery. We collected data on the year of publication, the reported postoperative results, and the outcome measures used to assess the results. Results: Sixty-one studies involving a total of 3987 surgical procedures were included. The average methodology score was 43.5 of 100. Methodological deficiencies were found with respect to five criteria: the type of study, description of the rehabilitation protocol, outcome criteria, outcome assessment, and subject selection process. Large variations in the reported outcome were seen within each treatment modality, and no significant differences were found between each kind of therapy (p = 0.11). The methodology score correlated positively with the level-of-evidence rating (r = 0.668, p < 0.0001), but there were large variations in the methodology score within each level. The linear regression analysis weighted by the number of patients demonstrated a negative yet not significant correlation between the methodology score and the results reported in nineteen studies with use of the Lysholm Scale (r = -0.29, p = 0.19). A total of twenty-seven different clinical outcome measurement scales were used to assess outcome. Conclusions: The generally low methodological quality found in the studies included in this analysis indicates that caution is required when interpreting results after surgical cartilage repair. Firm recommendations on which procedure to choose cannot be given at this time on the basis of these studies. More attention should be paid to methodological quality when designing, performing, and reporting clinical studies.