Quantifying the complexity of medical research

Motivation: A crucial phenomenon of our times is the diminishing marginal returns of investments in pharmaceutical research and development. A potential reason is that research into diseases is becoming increasingly complex, and thus more burdensome, for humans to handle. We sought to investigate whether we could measure research complexity by analyzing the published literature. Results: Through the text mining of the publication record of multiple diseases, we have found that the complexity and novelty of disease research has been increasing over the years. Surprisingly, we have also found that research on diseases with higher publication rate does not possess greater complexity or novelty than that on less-studied diseases. We have also shown that the research produced about a disease can be seen as a differentiated area of knowledge within the wider biomedical research. For our analysis, we have conceptualized disease research as a parallel multi-agent search in which each scientific agent (a scientist) follows a search path based on a model of a disease. We have looked at trends in facts published for diseases, measured their diversity and turnover using the entropy measure and found similar patterns across disease areas.