Evaluating instructional designs with mental workload assessments in university classrooms

Cognitive cognitive load theory (CLT) has been conceived for improving instructional design practices. Although researched for many years, one open problem is a clear definition of its cognitive load types and their aggregation towards an index of overall cognitive load. In Ergonomics, the situation is different with plenty of research devoted to the development of robust constructs of mental workload (MWL). By drawing a parallel between CLT and MWL, as well as by integrating relevant theories and measurement techniques from these two fields, this paper is aimed at investigating the reliability, validity and sensitivity of three existing self-reporting mental workload measures when applied to long learning sessions, namely, the NASA Task Load index, the Workload Profile and the Rating Scale Mental Effort, in a typical university classroom. These measures were aimed at serving for the evaluation of two instructional conditions. Evidence suggests these selected measures are reliable and their moderate validity is in line with results obtained within Ergonomics. Additionally, an analysis of their sensitivity by employing the descriptive Harrell-Davis estimator suggests that the Workload Profile is more sensitive than the Nasa Task Load Index and the Rating Scale Mental Effort for long learning sessions.

Automated summary

This paper investigates the reliability, validity, and sensitivity of three existing self-reporting mental workload measures in long learning sessions and compares them. The results suggest that these measures are reliable for evaluating instructional conditions and are consistent with findings in the field of ergonomics. Additionally, the Workload Profile is found to be more sensitive than the NASA Task Load Index and the Rating Scale Mental Effort in long learning sessions.