Reassessing the methods of medical record review studies in emergency medicine research

Study objective: An assessment of the methods of medical record review studies published in emergency medicine journals during a 5-year period ending in 1993 provided strategies for improvements. We assess and quantify the current methodologic quality of medical record review studies in emergency medicine journals using published guidelines and compare these results among journals and with those of 10 years previously. Methods: Independent, systematic searches of emergency medicine journals identified all medical record review studies published in 2003. Methodology assessments of each selected study were conducted independently by 2 other researchers, and disagreements were resolved by arbitration. Results: We identified 79 (14%) medical record review studies in 563 original research articles in 6 emergency medicine journals. The highest adherence to methodologic standards was found for sampling method (99%; 95% confidence interval [CI] 93% to 100%), and the lowest was for abstractor blinding to hypothesis (4%; 95% Cl 1% to 11%). Interobserver agreement for the 12 criteria ranged from 57% to 95%. A comparison of these results with those of 10 years ago revealed significant improvements in 3 of the 8 original criteria assessed: data abstraction forms, mentioning interobserver performance, and testing interobserver performance. Conclusion: Medical record review studies continue to comprise a substantial proportion of original research in the emergency medicine literature. Important improvements are noted in some criteria, but adherence remains below 50% for 7 of the 12 criteria assessed.