Designing Assessments and Assessing Designs in Virtual Educational Environments

This study used innovative assessment practices to obtain and document broad learning outcomes for a 15-hour game-based curriculum in Quest Atlantis, a multi-user virtual environment that supports school-based participation in socio scientific inquiry in ecological sciences. Design-based methods were used to refine and align the enactment of virtual narrative and scientific investigations to a challenging problem solving assessment and indirectly to achievement test items that were independent of the curriculum. In study one, one-sixth grade teacher used the curriculum in two of his classes and obtained larger gains in understanding and achievement than his two other classes, which used an expository text to learn the same concepts and skills. Further treatment refinements were carried out, and two forms of virtual formative feedback were introduced. In study two, the same teacher used the curriculum in all four of his classes; the revised curriculum resulted in even larger gains in understanding and achievement. Gains averaged 1.1 SD and 0.4 SD, respectively, with greater gains shown for students who engaged more with formative feedback. Principles for assessing designs and designing assessments in virtual environments are presented.