A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?

Scientific journals must deal with the following questions concerning the predictive validity of editorial decisions. Is the best scientific work selected from submitted manuscripts? Does selection of the best manuscripts also mean selecting papers that after publication show top citation performance within their fields? Taking the journal Angewandte Chemie International Edition as an example, this study proposes a new methodology for investigating whether manuscripts that are most worthy of publication are in fact selected validly. First, the influence on citation of the accepted and rejected but then published elsewhere manuscripts was appraised on the basis of percentile impact classes scaled in a subfield of chemistry and, second, the association between the decisions on selection and the influence on citation of the manuscripts was determined by using a multilevel logistic regression for ordinal categories. This approach has many advantages over methodologies that were used in previous research studies on the predictive validity of editorial selection decisions.