Reliability and validity of findings in ergonomics research

Evaluation of findings in ergonomics/human factors (E/HF) research suffers from misconceived assessments in terms of reliability and validity. Evaluation of E/HF studies published after 2000 confirms these observations. With an eye on these misconceived assessments, the present paper focuses on the consequences of various types of human involvement in (co-)shaping the phenomena to be evaluated. Issues addressed include the questionability of the inter-individual equivalence of findings, the defectiveness of the reliability coefficient as dispersion measure and the elusiveness of intended/presumed measurement as a validity criterion. These deficiencies are at odds with the zeal of E/HF authors to flag their findings as 'reliable' and 'valid'. In particular, positive evaluations of findings may show off as rhetoric. An evaluation procedure of consecutive constituents in a flowchart is proposed as an aid for appropriate evaluations. Various conditions are discussed that may encourage the adoption of correct procedures.