期刊
BEHAVIOR RESEARCH METHODS
卷 53, 期 4, 页码 1407-1425出版社
SPRINGER
DOI: 10.3758/s13428-020-01501-5
关键词
Accuracy; Experiment builder; Big data; Reaction time; MTurk; Online testing; System testing; Automated hardware testing; Psychophysics
资金
- MRC [MC_UP_A060_1103, MC_UU_00005/2] Funding Source: UKRI
- Medical Research Council [MC_UP_A060_1103, MC_UU_00005/2] Funding Source: Medline
Online behavioral research is booming due to its increasing ease of use and ability to quickly collect large samples. Researchers have conducted automated tests to understand how different web-building platforms, browsers, and operating systems impact experiment presentation and data quality. Findings show that modern web platforms provide reasonable accuracy and precision for display duration and manual response time, with no single platform standing out as the best in all features and conditions. Participants' demographics, technology, and software usage were also analyzed to provide context to the research findings.
Due to increasing ease of use and ability to quickly collect large samples, online behavioural research is currently booming. With this popularity, it is important that researchers are aware of who online participants are, and what devices and software they use to access experiments. While it is somewhat obvious that these factors can impact data quality, the magnitude of the problem remains unclear. To understand how these characteristics impact experiment presentation and data quality, we performed a battery of automated tests on a number of realistic set-ups. We investigated how different web-building platforms (Gorilla v.20190828, jsPsych v6.0.5, Lab.js v19.1.0, and psychoJS/PsychoPy3 v3.1.5), browsers (Chrome, Edge, Firefox, and Safari), and operating systems (macOS and Windows 10) impact display time across 30 different frame durations for each software combination. We then employed a robot actuator in realistic set-ups to measure response recording across the aforementioned platforms, and between different keyboard types (desktop and integrated laptop). Finally, we analysed data from over 200,000 participants on their demographics, technology, and software to provide context to our findings. We found that modern web platforms provide reasonable accuracy and precision for display duration and manual response time, and that no single platform stands out as the best in all features and conditions. In addition, our online participant analysis shows what equipment they are likely to use.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据