Learning on knowledge graph dynamics provides an early warning of impactful research

Biotechnology-related papers predicted to be of long-term impact are identified in a machine learning framework (DELPHI) that analyzes relationships among a range of features from the scientific literature over time. The scientific ecosystem relies on citation-based metrics that provide only imperfect, inconsistent and easily manipulated measures of research quality. Here we describe DELPHI (Dynamic Early-warning by Learning to Predict High Impact), a framework that provides an early-warning signal for 'impactful' research by autonomously learning high-dimensional relationships among features calculated across time from the scientific literature. We prototype this framework and deduce its performance and scaling properties on time-structured publication graphs from 1980 to 2019 drawn from 42 biotechnology-related journals, including over 7.8 million individual nodes, 201 million relationships and 3.8 billion calculated metrics. We demonstrate the framework's performance by correctly identifying 19/20 seminal biotechnologies from 1980 to 2014 via a blinded retrospective study and provide 50 research papers from 2018 that DELPHI predicts will be in the top 5% of time-rescaled node centrality in the future. We propose DELPHI as a tool to aid in the construction of diversified, impact-optimized funding portfolios.

Automated summary

This study introduces a machine learning framework called DELPHI to predict long-term impact research papers in the field of biotechnology. The framework provides early warning signals for impactful research by autonomously learning relationships among features in scientific literature. DELPHI has demonstrated the ability to correctly identify key biotechnologies and predict high-impact research papers in the future.