Comparing Three OnlineTesting Modalities: Using Static, Active, and Interactive OnlineTesting Modalities to Assess Middle School Students' Understanding of Fundamental Ideas and Use of Inquiry Skills Related to Ecosystems

Online testing holds much promise for assessing students' complex science knowledge and inquiry skills. In the current study, we examined the comparative effectiveness of assessment tasks and test items presented in online modules that used either a static, active, or interactive modality. A total of 1,836 students from the classrooms of 22 middle school science teachers in 12 states participated in the study as part of normal classroom activities. Students took assessments in the three different modalities on three consecutive days. The assessments tested key concepts about ecosystems and students' ability to use inquiry skills in an ecosystems context. Our in-depth analyses focused on how the different modalities elicited specific content knowledge of ecosystems (e.g., producers, consumers, predator-prey relationships) and specific inquiry skills (e.g., designing and interpreting experiments). We also investigated student use of technology supports, such as replaying animations or inspecting graphs. The results showed that the interactive modality enabled the testing of more complex reasoning and that additional experience working in the online environment improved student performance for all the modalities, especially for the interactive modality. Each of the three modalities provided useful information about students' understanding of ecosystems and related inquiry skills as well as their misconceptions. The study begins to build a knowledge base of what types of science knowledge and skills may be effectively measured in three different modalities of online assessment. (c) 2014 Wiley Periodicals, Inc. J Res Sci Teach 51: 523-554, 2014