Text Mining and Topic Modeling of Compendiums of Papers from Transportation Research Board Annual Meetings

The collective knowledge system has been advancing rapidly in the recent past. The digitalization of information in many online media such as blogs, social media, articles, webpages, images, audios, and videos provides an unprecedented opportunity for the extraction and identification of a knowledge trend. Prominent journal and conference proceedings usually contain extensive amounts of textual data that can be used to examine the research trends for various topics of interest and to understand how this research has helped in the advancement of a subject such as transportation engineering. The exploration of the unstructured contents in journal or conference papers requires sophisticated algorithms for knowledge extraction. This paper presents text mining techniques to analyze compendiums of papers published from TRB annual meetings, the largest and most comprehensive transportation conferences in the world. Topic models are algorithms designed to discover hidden thematic structure from massive collections of unstructured documents. This study used a popular topic model, latent Dirichlet allocation, to reveal research trends and interesting histories of the development of research by analyzing 15,357 compendiums of papers from 7 years (2008 to 2014) of TRB annual meetings.