What Do the Measures of Utterance Fluency Employed in Automatic Speech Evaluation (ASE) Tell Us About Oral Proficiency?

This paper explores what the measures of utterance fluency typically employed in Automatic Speech Evaluation (ASE), i.e. automated speaking assessments, tell us about oral proficiency. 60 Chinese learners of English completed the second part of the speaking section of IELTS and six tasks designed to measure the linguistic knowledge and processing assumed to underpin second language speech production. A sample of eight native speakers rated the learners' oral productions for functional adequacy. Analyses of the data confirm: (1) articulation rate, mid-clause pause frequency, and repetition frequency predict functional adequacy, (2) breadth of lexical knowledge is the main predictor of articulation rate as well as functional adequacy, (3) speed of syntactic processing predicts end-clause pause duration and speed of lexical processing predicts mid-clause pause duration, and (4) measures of utterance fluency together account for 60% of the variation in functional adequacy scores. These findings suggest that articulation rate best reflects overall functional adequacy. Moreover, other measures of utterance fluency reflect different areas of underlying knowledge and processing, opening up the possibility of automating diagnostic speaking tests.

Automated summary

This paper explores the effects of measures of utterance fluency typically used in Automatic Speech Evaluation on oral proficiency. Through a study on 60 Chinese learners of English, it was found that articulation rate, pause frequency, and repetition frequency can predict functional adequacy. Additionally, syntactic processing speed predicts pause duration at the end of clauses, while lexical processing speed predicts pause duration within clauses. Overall, measures of utterance fluency account for 60% of the variation in functional adequacy scores. These findings suggest that articulation rate best reflects overall functional adequacy, and other fluency measures reflect different underlying knowledge and processing areas, indicating the potential for automating diagnostic speaking tests.