Assessment of the consistency of medicinal chemists in reviewing sets of compounds

Medicinal chemists are frequently asked to review lists of compounds to assess their drug- or leadlike nature and to evaluate the suitability of lead compounds based on their attractiveness and/or synthetic feasibility as a basis for launching a drug-discovery campaign. It is often felt that one medicinal chemist's opinion is as good as any other, but is it? In an attempt to answer this question, an experiment was performed in conjunction with a recent compound acquisition program (CAP) conducted at Pharmacia. Historically, the CAP included a review of many thousands of compounds by medicinal chemists who eliminate anything deemed undesirable for any reason. In a review conducted in 2002, about 22 000 compounds requiring review by medicinal chemists were broken down into 11 lists of approximately 2000 compounds each. Unknown to the medicinal chemists, a subset of 250 compounds, previously rejected by a very experienced senior medicinal chemist, was added to each of the lists. Most of the 13 medicinal chemists who participated in this process reviewed two lists, although some only reviewed a single list and one reviewed three lists. Those compounds that were deemed unacceptable were recorded and tabulated in various ways to assess the consistency of the reviews. It was found that medicinal chemists were not very consistent in the compounds they rejected as being undesirable. The inconsistency arises from the subjective analysis that all humans utilize when considering data sets of any kind. This has important implications for pharmaceutical project teams where individual medicinal chemists review lists of primary screening hits to identify those compounds suitable for follow-up. Once a compound is removed from a list, it and other structurally similar compounds are effectively removed from further consideration. This can also have an impact on computational chemists who are developing models for assessing the desirability or attractiveness of different classes of compounds for lead discovery.