Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign

Background: In postgraduate specialist training, workplace assessments are expected to provide the information required for decisions on trainee progression. Research suggests that meeting this expectation can be difficult in practice, which has led to the development of informal processes, or 'shadow systems' of assessment. Rather than rejecting these informal approaches to workplace assessment, we propose borrowing from sociology the concept of 'desire paths' to legitimise and strengthen these well-trodden approaches. We asked what information about trainees is currently used or desired by those charged with making decisions on trainee progression, and how is it obtained? Methods: We undertook a qualitative study with thematic analysis of semi-structured interviews of supervisors of training across Australia and New Zealand. Results: From 21 interviews, we identified four interrelated themes, the first being the local context of training sites. The other three themes represent dilemmas in the desire for authentic and representative information about the trainee: 1) how the process of gathering and documenting information can filter, transform, or limit the original message; 2) deciding when possible trainee deviation from performance norms warrants a closer look; and 3) how transparent vs covert information gathering affects the information supervisors will provide, and how control over assessment is distributed between trainee and supervisor. Conclusion: From these themes, we propose a set of design principles for future workplace assessment. Understanding the reasons desire paths exist can inform future assessment redesign, and may address the current disjunct between the formal workplace assessment system and what happens in practice.

Automated summary

This study aims to understand the current use and desired information about trainees by decision-makers and the methods of obtaining this information. Through interviews with 21 training supervisors in Australia and New Zealand, four interrelated themes were identified, including the local context of training sites, the impact of the information gathering and documentation process, when to pay attention to possible deviations from performance norms, the effects of transparent and covert information gathering on assessment outcomes, and the distribution of assessment control between trainees and supervisors. Based on these themes, a set of design principles for future workplace assessment is proposed.