Statistics, truth, and error reduction in sport and exercise sciences

Academics have a responsibility to ensure that their research findings are as truthful as possible. In every issue of a scientific journal, a large number of significance tests are reported (usually using P < 0.05). Of course, most of these results will be true/correct. Unfortunately, due to the nature of sampling, researchers will occasionally make errors, often referred to as type I (probability = alpha) and type II (probability = beta) errors. The power of a test (1-beta) is the probability of correctly rejecting a false null hypothesis - that is, correctly detecting a real or true effect. Factors that are known to influence power include: (1) the level of significance (alpha), (2) the size of the difference or relationship in the population (the effect), (3) the sample size, and (4) unexplained error variance. As researchers, we have little control over most of these factors. The one factor that we have some influence over, however, is the ability to reduce the unexplained error variance. In the present article, we describe a range of methods that will increase the probability that a researcher has correctly identified a real effect by increasing the power of their statistical tests. Such methods will include ways of designing experiments to reduce error and uncertainty. The use of blocking and randomized block designs will reduce unexplained error, such as adopting matched or repeated-measures designs rather than using independent observations. The other method of reducing unexplained errors is to adopt more appropriate (e.g. biologically correct) models and checking the distribution assumptions associated with such models. In conclusion, researchers are responsible for maximizing the likelihood that their results are as accurate and truthful as possible. By carefully planning their experiments and adopting appropriate models, researchers are more likely to publish their findings with a greater degree of confidence, but not certainty.