Unsupervised clinical relevancy ranking of structured medical records to retrieve condition-specific information in the emergency department

Background: Decision making in the Emergency Department (ED) requires timely identification of clinical information relevant to the complaints. Existing information retrieval solutions for the electronic health record (EHR) focus on patient cohort identification and lack clinical relevancy ranking. We aimed to compare knowledge-based (KB) and unsupervised statistical methods for ranking EHR information by relevancy to a chief complaint of chest or back pain among ED patients. Methods: We used Pointwise-mutual information (PMI) with corpus level significance adjustment (cPMId), which modifies PMI to reward co-occurrence patterns with a higher absolute count. cPMId for each pair of medication/problem and chief complaint was estimated from a corpus of 100,000 un-annotated ED encounters. Five specialist physicians ranked the relevancy of medications and problems to each chief complaint on a 0-4 Likert scale to form the KB ranking. Reverse chronological order was used as a baseline. We directly compared the three methods on 1010 medications and 2913 problems from 99 patients with chest or back pain, where each item was manually labeled as relevant or not to the chief complaint, using mean average-precision. Results: cPMId out-performed KB ranking on problems (86.8% vs. 81.3%, p < 0.01) but under-performed it on medications (93.1% vs. 96.8%, p < 0.01). Both methods significantly outperformed the baseline for both medications and problems (71.8% and 72.1%, respectively, p < 0.01 for both comparisons). The two complaints represented virtually completely different information needs (average Jaccard index of 0.008). Conclusion: A fully unsupervised statistical method can provide a reasonably accurate, low-effort and scalable means for situation-specific ranking of clinical information within the EHR.

Automated summary

The study compared knowledge-based and unsupervised statistical methods for ranking clinical information in Emergency Department patients with chest or back pain complaints. The results showed that the unsupervised statistical method outperformed the knowledge-based ranking for problems, but underperformed for medications.