Consistent or not? An investigation of using Pull Request Template in GitHub

Context: The arbitrary usage of pull requests in GitHub may bring many issues such as incomplete, verbose, and duplicated descriptions, which hinder the development and maintenance of the project. Thus, GitHub proposed the Pull Request Template (PRT) in 2016 so that developers could edit the pull request in a relevant consistent manner. However, whether the PRT has been widely applied to GitHub and what impact it might bring remain little known. Such uninformed cases may affect the efficiency of cooperative development. Objective: In this work, we conduct an empirical study on large-scale repositories to explore whether the PRT has been widely applied and what impact it can bring to the GitHub community. Method: This work aims to answer four research questions. The first is a statistical experiment with the aim of analyzing the current status of PRTs. The second is an explored experiment, which aims at probing which repositories are suitable to adopt the PRT. The third is the measurement evaluation experiment, focusing on discussing what impact the PRT can bring. The last is an online survey to explain why few PRTs have been adopted. Notably, both the second and third questions are conducted a mixed quantitative and qualitative analysis. Results and conclusion: In this work, we find that only 1.2% of repositories contain the PRT in GitHub, and such repositories are mostly in high popularity and contain a large number of PRs. Besides, contributors are willing to accept the PRT that requires pivotal information, including description, test, and check_list. Meanwhile, the PRT can assist developers to manage repositories, reflecting in less time for reviewing, fewer duplicated pull requests, and almost non-existentially invalid comments. Finally, we survey 527 well-reputed developers to explain why few repositories adopt the PRT, and further provide some actionable suggestions.

Automated summary

The study found that only 1.2% of GitHub repositories contain the PRT, mainly in high popularity and with a large number of PRs. Contributors are willing to accept the PRT that requires key information, while it also helps to manage repositories, leading to shorter review time, fewer duplicated pull requests, and minimal invalid comments.