THE EFFECT DIRECTION SHOULD BE TAKEN INTO ACCOUNT WHEN ASSESSING SMALL-STUDY EFFECTS

ObjectiveStudies with statistically significant results are frequently more likely to be pub-lished than those with non-significant results. This phenomenon leads to publi-cation bias or small-study effects and can seriously affect the validity of the con-clusion from systematic reviews and meta-analyses. Small-study effects typically appear in a specific direction, depending on whether the outcome of interest is beneficial or harmful, but this direction is rarely taken into account in conven-tional methods.MethodsWe propose to use directional tests to assess potential small-study effects. The tests are built on a one-sided testing framework based on the existing Egger's regression test. We performed simulation studies to compare the proposed one-sided regression tests, conventional two-sided regression tests, as well as two other competitive methods (Begg's rank test and the trim-and-fill method). Their performance was measured by type I error rates and statistical power. Three real -world meta-analyses on measurements of infrabony periodontal defects were also used to examine the various methods' performance.Results Based on simulation studies, the one-sided tests could have considerably higher statistical power than competing methods, particularly their two-sided counter-parts. Their type I error rates were generally controlled well. In the case study of the three real-world meta-analyses, by accounting for the favored direction of effects, the one-sided tests could rule out potential false-positive conclusions about small-study effects. They also are more powerful in assessing small-study effects than the conventional two-sided tests when true small-study effects likely exist.Conclusion We recommend researchers incorporate the potential favored direction of ef-fects into the assessment of small-study effects.

Automated summary

Directional tests can be used to assess small-study effects, providing a more accurate evaluation of the validity of research findings. These tests showed better performance in simulation studies and real-world case studies, effectively ruling out false conclusions and assessing small-study effects more powerfully.