Committee peer review at an international research foundation: predictive validity and fairness of selection decisions on post-graduate fellowship applications

We investigated committee peer review for awarding long-term fellowships to post-graduate researchers as practised by the Boehringer Ingelheim Fonds (BIF) - a foundation for the promotion of basic research in biomedicine. The most important aspect of our study was to investigate the predictive validity of the procedure, i.e. whether the foundation achieves its aim to select as fellowship recipients the best junior scientists. Our bibliometric analysis shows that this is indeed the case and that the selection procedure is thus highly valid. With regard to fairness of the procedure, we analysed the extent to which the foundation's Board of Trustees' practice of reviewing the applications in alphabetic order when making final selection decisions has an influence on the decisions that they make. A statistically significant influence of the postulated bias variable could be observed, but the effect size was small.