Assessing the Assessment-Developing and Deploying a Novel Tool for Evaluating Clinical Notes' Diagnostic Assessment Quality

BackgroundAmbulatory diagnostic errors are increasingly being recognized as an important quality and safety issue, and while measures of diagnostic quality have been sought, tools to evaluate diagnostic assessments in the medical record are lacking.ObjectiveTo develop and test a tool to measure diagnostic assessment note quality in primary care urgent encounters and identify common elements and areas for improvement in diagnostic assessment.DesignRetrospective chart review of urgent care encounters at an urban academic setting.ParticipantsPrimary care physicians.Main MeasuresThe Assessing the Assessment (ATA) instrument was evaluated for inter-rater reliability, internal consistency, and findings from its application to EHR notes.Key ResultsATA had reasonable performance characteristics (kappa 0.63, overall Cronbach's alpha 0.76). Variability in diagnostic assessment was seen in several domains. Two components of situational awareness tended to be well-documented (Don't miss diagnoses present in 84% of charts, red flag symptoms in 87%), while Psychosocial context was present only 18% of the time.ConclusionsThe ATA tool is a promising framework for assessing and identifying areas for improvement in diagnostic assessments documented in clinical encounters.

Automated summary

The study aimed to develop and test a tool to measure the quality of diagnostic assessment notes in primary care urgent encounters and identify common elements and areas for improvement. The results showed variability in diagnostic assessment in several domains, with two components of situational awareness being well-documented and psychosocial context being present only 18% of the time.