Statistical Comparison between In-Person and Online General Chemistry Exam Outcomes: A COVID-Induced Case Study

In spring 2020, the chemical education community facedan abrupttransition from in-person to online classes, which also necessitatedonline assessments. Building upon an existing three-semester study(F17, S19, and F19) using Rasch modeling and classical testing theoryto improve in-person multiple choice exams, this study investigatesthe impact of online exams (F20, F21, and F22) on assessment qualityand student performance in an undergraduate General Chemistry II class.The Cronbach's alpha and fraction of very good/good questionswere found to dramatically increase across the first two semesters(F17 and S19) and then largely plateaued for subsequent exams, regardlessof in-person or online test administration. Through the use of linkingquestions (i.e., repeated questions from semester to semester) andequating procedures, the results indicated that (1) there was notan obvious or uniform increase or decrease in the exam quality orstudent performance when switching from in-person to online examsand (2) there was no evidence for an increased prevalence of cheatingin the unproctored online exam relative to the prior in-person exams.While this data set is not sufficient to make any universal claims,this case study's outcomes suggest that concerns about increasedcheating on unproctored online exams are not inherently founded.

Automated summary

In spring 2020, the chemical education community faced a sudden transition to online classes and assessments. This study examines the impact of online exams on assessment quality and student performance in a General Chemistry II class. The results indicate that the quality of exams and student performance remained consistent regardless of in-person or online administration, and there was no evidence of increased cheating in online exams compared to in-person exams. Although these findings cannot be universally applied, they suggest that concerns about cheating in unproctored online exams may not be valid.