Predicting Learning in a Multi-component Serious Game

The current study investigated predictors of shallow versus deep learning within a serious game known as Operation ARA. This game uses a myriad of pedagogical features including multiple-choice tests, adaptive natural language tutorial conversations, case-based reasoning, and an E-text to engage students. The game teaches 11 topics in research methodology across three distinct modules that target factual information, application of reasoning to specific cases, and question generation. The goal of this investigation is to discover predictors of deep and shallow learning by blending Evidence-Centered Design (ECD) with educational data mining. In line with ECD, time-honored cognitive processes or behaviors of time-on-task, discrimination, generation, and scaffolding were selected because there is a large research history supporting their importance to learning. The study included 192 college students who participated in a pretest-interaction-posttest design. These data were used to discover the best predictors of learning across the training experiences. Results revealed distinctly different patterns of predictors of deep versus shallow learning for students across the training environments of the game. Specifically, more interactivity is important for environments contributing to shallow learning whereas generation and discrimination is more important in training environments supporting deeper learning. However, in some training environments the positive impact of generation may be at the price of decreased discrimination.