Evaluation of a drug and alcohol safety education program in aviation using interrupted time series and the Kirkpatrick framework

Compliance with drug and alcohol regulations are required by 14 CFR Part 120/ and 49 CFR Part 40. These regulations affect approximately 7200 aviation-related companies and their associated services. Consequences for noncompliance can include loss of revenue from imposition of civil penalties, suspension, or revocation of the company's certificate to conduct business. Front End Analysis (FEA) was conducted to determine specific performance problems and provide five tailored educational interventions to address them. Program evaluation was conducted using Interrupted Time Series (ITS) modeling. Results showed significant (p < .05) decreases in nonconformities across all five models with small to moderate effect sizes. Based on the relative effects, values for reductions in civil penalty costs between 16% and 47%, were predicted. Actual sanction reductions from the pre-to-post-intervention periods were confirmed to be 24.21%. The study supported the efficacy of the ITS approach for implementation of level four Kirkpatrick evaluation.