Automated Guidance for Thermodynamics Essays: Critiquing Versus Revisiting

Middle school students struggle to explain thermodynamics concepts. In this study, to help students succeed, we use a natural language processing program to analyze their essays explaining the aspects of thermodynamics and provide guidance based on the automated score. The 346 sixth-grade students were assigned to either the critique condition where they criticized an explanation or the revisit condition where they reviewed visualizations. Within each condition, the student was assigned one of two types of tailored guidance based on the sophistication of their original essay. Both forms of guidance led to significant improvement in student understanding on the posttest. Guidance was more effective for students with low prior knowledge than for those with high prior knowledge (consistent with regression toward the mean). However, analysis of student responses to the guidance illustrates the value of aligning guidance with prior knowledge. All students were required to revise their essay as an embedded assessment. While effective, teachers involved in this study reported that revising is resisted by students and does not align with typical, vocabulary-focused classroom writing activities.