Finding Trends in Software Research

Text mining methods can find large scale trends within research communities. For example, using stable Latent Dirichlet Allocation (a topic modeling algorithm) this study found 10 major topics in 35,391 SE research papers from 34 leading SE venues over the last 25 years (divided, evenly, between conferences and journals). Out study also shows how those topics have changed over recent years. Also, we note that (in the historical record) mono-focusing on a single topic can lead to fewer citations than otherwise. Further, while we find no overall gender bias in SE authorship, we note that women are under-represented in the top-most cited papers in our field. Lastly, we show a previously unreported dichotomy between software conferences and journals (so research topics that succeed at conferences might not succeed at journals, and vice versa). An important aspect of this work is that it is automatic and quickly repeatable (unlike prior SE bibliometric studies that used tediously slow and labor intensive methods). Automation is important since, like any data mining study, its conclusions are skewed by the data used in the analysis. The automatic methods of this paper make it far easier for other researchers to re-apply the analysis to new data, or if they want to use different modeling assumptions.

Automated summary

Text mining methods can discover large-scale trends within research communities. This study, using stable Latent Dirichlet Allocation, identified 10 major research topics and tracked their changes over the past 25 years in 35,391 software engineering papers. It also revealed the potential negative impact of focusing too much on a single topic and the under-representation of women in highly cited papers. Additionally, the study highlighted an unreported dichotomy between software conferences and journals, suggesting different levels of success for research topics in each.