Assessing the quality of clinical data in a computer-based record for calculating the Pneumonia Severity Index

Objective: This study examined whether clinical data routinely available in a computerized patient record (CPR) can be used to drive a complex guideline that supports physicians in real time and at the point of care in assessing the risk of mortality for patients with community-acquired pneumonia. Setting: Emergency department of a tertiary-care hospital. Design: Retrospective analysis with medical chart review. Patients: All 241 inpatients during a 17-month period (Jun 1995 to Nov 1996) who presented to the emergency department and had a primary discharge diagnosis of community-acquired pneumonia. Methods/Main Outcome Measures: The 20 guideline variables were extracted fram the CPR (HELP System) and the paper chart. The risk score and the risk class of the Pneumonia Severity Index were computed using data from the CPR alone and from a reference standard of all data available in the paper chart and the CPR at the time of the emergency department encounters. Availability and concordance were quantified to determine data quality. The type and cause of errors were analyzed depending on the source and format of the clinical variables. Results: Of the 20 guideline variables, 12 variables were required to be present for every computer-charted emergency department patient, seven variables were required for selected patients only, and one variable was not typically available in the HELP System during a patient's encounter. The risk class was identical for 86.7 percent of the patients. The majority of patients with different risk classes were assigned too low a risk class. The risk scores were identical for 72.1 percent of the patients. The average availability was 0.99 for the data elements that were required to be present and 0.79 for the data elements that were not required to be present. The average concordance was 0.98 when all a patient's variables were taken into account. The cause of error was attributed to the nurse charting in 77 percent of the cases and. to the computerized evaluation in 23 percent. The type of error originated from the free-text fields in 64 percent, from coded fields in 21 percent, from vital signs in 14 percent and from laboratory results in 1 percent. Conclusion: From a clinical perspective, the current level of data quality in the HELP System supports the automation and the prospective evaluation of the Pneumonia Severity Index as a computerized decision support tool.