Perspective: The ACGME Toolbox: Half Empty or Half Full?

The Accreditation Council for Graduate Medical Education Outcome Project changed the currency of accreditation from process and structure to outcomes. Residency program directors must document their residents' competence in six general dimensions of practice. A recent systematic review, published in the March 2009 issue of Academic Medicine, concluded that the instruments currently available are psychometrically inadequate for evaluating residents in five of the six competencies. In this perspective, the authors refute the findings of this earlier review. They demonstrate that the review's search strategy was limited, failing to capture many important evaluation studies. They also question the appropriateness of the analysis of the included articles, which focused, to the exclusion of other important properties, on an instrument's ability to discriminate among residents' performance in the six competencies. Finally, the authors argue that the problem is not the lack of adequate evaluation instruments but, rather, the inconsistent use and interpretation of such instruments by unskilled faculty. They urge the graduate medical education community-if it is to realize the promise of competency-based education-to invest in training for faculty evaluators rather than waiting for new instruments.