期刊
INTERNATIONAL JOURNAL OF SOFTWARE ENGINEERING AND KNOWLEDGE ENGINEERING
卷 32, 期 7, 页码 947-970出版社
WORLD SCIENTIFIC PUBL CO PTE LTD
DOI: 10.1142/S0218194022500358
关键词
Open source; software document; software reproducibility; scholarly paper; information retrieval
类别
资金
- 13th Five-Year Plan project Artificial Intelligence and Language of State Language Commission of China [WT135-38]
Source code is crucial for researchers to reproduce and replicate the results of AI papers. To address the labor-intensive and time-consuming task of manual collection, researchers propose a method to automatically identify and extract source code from papers. They find that 20.5% of top AI conference papers have available source code, but 8.1% of the source code repositories are no longer accessible.
Source code is essential for researchers to reproduce the methods and replicate the results of artificial intelligence (AI) papers. Some organizations and researchers manually collect AI papers with available source code to contribute to the AI community. However, manual collection is a labor-intensive and time-consuming task. To address this issue, we propose a method to automatically identify papers with available source code and extract their source code repository URLs. With this method, we find that 20.5% of regular papers of 10 top AI conferences published from 2010 to 2019 are identified as papers with available source code and that 8.1% of these source code repositories are no longer accessible. We also create the XMU NLP Lab README Dataset, the largest dataset of labeled README files for source code document research. Through this dataset, we have discovered that quite a few README files have no installation instructions or usage tutorials provided. Further, a large-scale comprehensive statistical analysis is made for a general picture of the source code of AI conference papers. The proposed solution can also go beyond AI conference papers to analyze other scientific papers from both journals and conferences to shed light on more domains.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据