A realistic empirical evaluation of the costs and benefits of UML in software maintenance

The Unified Modeling Language (UML) is the de facto standard for object-oriented software analysis and design modeling. However, few empirical studies exist which investigate the costs and evaluate the benefits of using UML in realistic contexts. Such studies are needed so that the software industry can make informed decisions regarding the extent to which they should adopt UML in their development practices. This is the first controlled experiment that investigates the costs of maintaining and the benefits of using UML documentation during the maintenance and evolution of a real nontrivial system, using professional developers as subjects, working with a state-of-the-art UML tool during an extended period of time. The subjects in the control group had no UML documentation. In this experiment, the subjects in the UML group had, on average, a practically and statistically significant 54 percent increase in the functional correctness of changes (p = 0.03) and an insignificant 7 percent overall improvement in design quality (p = 0.22), though a much larger improvement was observed on the first change task ( 56 percent), at the expense of an insignificant 14 percent increase in development time caused by the overhead of updating the UML documentation (p = 0.35).