A Survey Evaluation of the Reliability of Visual Inspection and Functional Analysis Graphs

Visual inspection is the primary method used to analyze graphed behavioral data produced by functional analyses of problem behavior. The purpose of this study was to examine rater reliability of functional analysis graphs using visual inspection. Forty-three participants responded to a one-time anonymous survey (N = 454) mailed to graduate programs accredited in applied behavior analysis (N = 11). Respondents were instructed to classify single-subject data arrays depicted in multielement graphs from published studies. Classification was based on function categories by Hagopian, Fisher, Thompson, and Owen-DeSchryver. Three indices based on overall inter-rater agreement, the consistency of individual raters across graphs, and the aggregate performance of raters per graph were calculated and compared. Results for all three indices of rater performance were moderate to low. Results confirm prior preliminary work indicating relatively low levels of reliability and are discussed in relation to research, training, and practice issues.