A novel machine learning approach for evaluation of public policies: An application in relation to the performance of university researchers

Research has become the main reference point for academic life in modern universities. Research incentives have been a controversial issue, because of the difficulty of identifying who are the main beneficiaries and what are the long-term effects. Still, new policies including financial incentives have been adopted to increase the research output at all possible levels. Little literature has been devoted to the response to those incentives. To bridge this gap, we carry out our analysis with data of a six years program developed in Madrid (Spain). Instead of using a traditional econometric approach, we design a machine learning multilevel model to discover on whom, when, and for how long those policies have an effect. The empirical model consists of an automated nested longitudinal clustering (ANLC) performed in two stages. Firstly, it performs a stratification of academics, and secondly, it performs a longitudinal segmentation for each group. The second part considers the researchers' sociodemographic, academic information and the evolution of their performance over time in the form of the annual percentage variation of their marks over the period. The new methodology, whose robustness is tested with a multilayer perceptron artificial neural network with a back-propagation learning algorithm, shows that tenure track researchers present a better response to incentives than tenured researches, and also that gender plays an important role in academia. These discoveries are relevant to administrations and universities for understanding the productivity of academics working under long-term incentive-based programs, the drawbacks and the inequalities for maximizing the generation of knowledge.