Code Quality: Examining the Efficacy of Automated Tools

A number of software tools have been designed to determine code quality. These automated tools can examine a computer program and provide a score based on a designed set of software metrics. In this exploratory study, we examined the quality of code written by students in an introductory Python programming course on one programming assignment. Each student submission was evaluated using an automated tool, Pylint, a code analyzer widely used by the Python community. The instructor also graded these submissions using predefined rubrics that evaluated code logic, syntax and style. We compared the two code quality scores. We found that Pylint does a good job of identifying errors, and formatting issues. But, the Pylint scores were lower than those provided manually by instructors.