The Mathematical Meaninglessness of the NASA Task Load Index: A Level of Measurement Analysis

Human mental workload can profoundly impact human performance and is thus an important consideration in the design and operation of many systems. The standard method for assessing human mental workload is the NASA Task Load Index (NASA-TLX). This involves a human operator subjectively rating a task based on six dimensions. These dimensions are combined into a single workload score using one of two methods: scaling and summing the dimensions (where scales are derived from a paired comparisons procedure) or averaging dimensions together. Despite its widespread use, the level of measurement of NASA-TLX's dimensions and its computed workload score has not been investigated. Additionally, nobody has researched whether NASA-TLX's two approaches for computing overall workload are mathematically meaningful with respect to the constituent dimensions' levels of measurement. This is a serious deficiency. Knowing what the level of measurement is for NASA-TLX scores will determine what mathematics can be meaningfully applied to them. Furthermore, if NASA-TLX workload syntheses are mathematically meaningless, then the measure lacks construct validity. The research presented in this article used a previously developed method to evaluate the level of measurement of NASA-TLX workload and its dimensions. Results show that the dimensions can, in most situations, be treated as interval in population analyses and ordinal for individuals. Our results also suggest that the methods for combining dimensions into workload scores are meaningless. We recommend that analysts evaluate the dimensions of NASA-TLX without combining them.

Automated summary

Human mental workload is crucial in designing and operating systems. The NASA Task Load Index (NASA-TLX) is the standard method for assessing this workload, but its dimensions and computed workload score have not been thoroughly investigated. The research presented in this article evaluated the level of measurement of NASA-TLX workload and dimensions, concluding that the dimensions can be treated as interval in population analyses and ordinal for individuals. However, the methods for combining dimensions into workload scores are found to be meaningless.