The on-line assessment of metacognitive skills in a computerized learning environment

Metacognitive skills regulate and control learning processes. For assessing metacognitive skills in learners, on-line assessment is required during actual task performance. An unobtrusive on-line method is the analysis of learner activities that are registered in logfiles of computerized tasks. As logfiles cannot reflect the learner's metacognitive considerations for enacting specific activities, logfile analysis should be validated against other on-line methods. Also, external validity of logfile measures needs to be established with related measures, such as learning performance. Fifty-two second-year students (13 years) from pre-academic education performed a computerized inductive-learning task. Traces of learner activities were stored in logfiles and automatically scored on indicators of metacognitive skills. Afterwards, participants completed learning-performance posttests. Results show high convergent validity between logfile indicators and human judgments of traced learner activities. Moreover, external validity was obtained for logfile measures in relation to learning performance (but not regarding participants' IQ scores). Implications for logfile analysis are discussed. (C) 2013 Elsevier Inc. All rights reserved.