Designing Automated Guidance to Promote Productive Revision of Science Explanations

Supporting students to revise their written explanations in science can help students to integrate disparate ideas and develop a coherent, generative account of complex scientific topics. Using natural language processing to analyze student written work, we compare forms of automated guidance designed to motivate productive revision and help students integrate their understanding of science. Research shows the benefit of providing timely, transparent guidance to students and identifies some challenges. Specifically, (a) students often believe online guidance is generic rather than adapted to their response; and (b) students do not always engage effortfully with online guidance to improve their written responses. We conducted two studies to address these challenges. In Study 1, we created transparent guidance that clarified how the computer personalizes guidance based on the student response. We hypothesized that transparent guidance would be especially valuable for low prior knowledge students who might expect the computer guidance to be too difficult. We found that transparent guidance had a greater impact than typical guidance on low prior knowledge student revisions, suggesting that student beliefs about how guidance is designed influence their performance. In Study 2, implemented in six schools, we compared two specific guidance strategies: revisiting evidence and planning writing changes. We found that both revisiting and planning guidance resulted in significant improvement in student knowledge integration, although neither guidance strategy showed a significant advantage over the other. In addition, we found that the form of guidance interacted with school, suggesting that teacher practices could reinforce a specific guidance strategy. These results illustrate ways to design guidance to strengthen student understanding of science. They raise important questions about when to encourage revisiting, how to design instruction focused on planning, and how to instill a lifelong practice of engaging in iterative refinement of scientific explanations.