Measuring professionalism: A review of studies with instruments reported in the literature between 1982 and 2002

Purpose To describe the measurement properties of instruments reported in the literature that faculty might use to measure professionalism in medical students and residents. Method The authors reviewed studies published between 1982 and 2002 that had been located using Medline and four other databases. A national panel of 12 experts in measurement and research in medical education extracted data from research reports using a structured critique form. Results A total of 134 empirical studies related to the concept of professionalism were identified. The content of 114 involved specific elements of professionalism, such as ethics, humanism, and multiculturalism, or associated phenomena in the educational environment such as abuse and cheating. Few studies addressed professionalism as a comprehensive construct (11 studies) or as a distinct facet of clinical competence (nine studies). The purpose of 109 studies was research or program evaluation, rather than summative or formative assessment. Sixty five used self-administered instruments with no independent observation of the participants' professional behavior. Evidence of reliability was reported in 62 studies. Although content validity was reported in 86 studies, only 34 provided strong evidence. Evidence of concurrent or predictive validity was provided in 43 and 16 studies, respectively Conclusions There are few well-documented studies of instruments that can be used to measure professionalism in formative or summative evaluation. When evaluating the tools described in published research it is essential for faculty to look critically for evidence related to the three fundamental measurement properties of content validity, reliability, and practicality.