Which statistical tests to use when analyzing NASA TLX? The NASA Task Load Index (NASA-TLX) is a subjective, multidimensional assessment tool that rates perceived workload. A set of different scales (Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, Frustration) are combined into one measure ranging from 0 to 100, the task load index. 

Which statistical tests can be used to compare these test results? I am particularly concerned with whether or not the task load index could be considered an interval variable, and thus whether, for example, a simple student t-test could be used. In particular, is the NASA-TLX scalar invariant? Can a difference between two means (e.g., 50 and 60) be interpreted as equal to a difference between two other means (e.g., 10 and 20)?

If not, what are the consequences of using a wrong test? What is a suitable test?
In Stevens' levels of measurement framework, the NASA-TLX is an ordinal scale, not an interval scale, because there is no way to know a priori how much "workload" each point corresponds to. In other words, we can only know how much workload a point corresponds to after observing the data from our sample. Indeed, Hart's own review of the NASA-TLX suggests that workload is "ill-defined" (Hart, 2006). 

While levels of measurement often seems to be taught in behavioral and social science as accepted fact, however, the levels of measurement framework is quite controversial among methodologists. It is a useful enough heuristic taxonomy for didactic purposes, but it is not known to be inferentially meaningful. Analytic relatives of the t-test (i.e., least-squares/maximum likelihood regression) exist for all four types, so the type of scale should not necessarily concern you too much here. I have not encountered it outside the behavioral sciences.

Statistical inference is essentially a mathematical pathfinder that formally relates a specific question to a specific answer by way of the evidence. It is therefore impossible to say what a suitable test would be without knowing the specific question(s)―different analyses test different questions. For the same data set, one analysis may be meaningful (i.e., discriminates between conceivable answers) for one question, but not another; for another analysis, this may be reversed. The consequence of using a wrong test is that you effectively answer a question you did not ask, while not answering the question you did ask.

References


Hart SG. NASA-Task Load Index (NASA-TLX): 20 years later. Paper presented at the HFES; San Francisco, CA. 2006.