Cumulative sum (Cusum) analysis provides an objective measure of competency during training in endoscopic retrograde cholangio-pancreatography (ERCP)

Background and Aim: Endoscopic retrograde cholangio-pancreatography (ERCP) is a complex and technically challenging procedure with a risk of significant complications. Although recommendations exist as to how many ERCPs should be performed before a trainee is deemed competent they lack objective measures of outcome. Cumulative sum (Cusum) analysis allows continuous monitoring of a proceduralist's performance to ensure acceptable outcomes are being achieved. The aim of the present study was to assess a trainee's progress with ERCP using Cusum analysis. Method: A single trainee performed one supervised ERCP list per week between June 2005 and May 2008. Prospective data were collected and analysed by intention to treat using Cumulative failure charting and Cusum analysis. The binary outcome measure was successful cannulation in patients with a native sphincter. Acceptable and unacceptable cannulation failure rates were set at 20% and 35% respectively, with Type I and II error rates set at 0.10. Results: A total of 290 ERCPs were performed. Successful cannulation was achieved in 168 of 238 (70.6%) patients with intact biliary sphincters. Although cumulative failure charting suggested the trainee had not yet achieved satisfactory performance, Cusum analysis with sequential probability ratio testing indicated that an acceptable outcome had been achieved for the last 179 consecutive ERCPs. Conclusion: Cusum analysis enables sensitive and continuous monitoring of a trainee's performance to objectively determine competency. Wider and systematic use may enable appropriate benchmarks to be identified and more objective assessment of a trainee's experience.