Eliciting and using expert knowledge in metrology

The expression of uncertainty has hitherto been seen as an add-on-first an estimate is obtained and then uncertainty in that estimate is evaluated. We argue that quantification of uncertainty should be an intrinsic part of measurement and that the measurement result should be a probability distribution for the measurand. Full quantification of uncertainties in measurement, recognizing and quantifying all sources of uncertainty, is rarely simple. Many potential sources of uncertainty can effectively only be quantified by the application of expert judgement. Scepticism about the validity or reliability of expert judgement has meant that these sources of uncertainty have often been overlooked, ignored or treated in a qualitative, narrative way. But the consequence of this is that reported expressions of uncertainty regularly understate the true degree of uncertainty in measurements. This article first discusses the concept of quantifying uncertainty in measurement, and then considers some of the areas where expert judgement is needed in order to quantify fully the uncertainties in measurement. The remainder of the article is devoted to describing methodology for eliciting expert knowledge.