Investigating instrument-based pragmatic variability: Effects of enhancing discourse completion tests

A significant dilemma in sociolinguistic research concerns the methods used to collect data, the validity of different types of data and,'...their adequacy to approximate the authentic performance of linguistic action' (Kasper and Dahl 1991: 215). Manes and Wolf son (1981) claimed that the most authentic data in sociolinguistic research is spontaneous speech gathered by ethnographic observation. Difficulties in relying on this method are well-documented (Blum-Kulka et al. 1989, Aston 1995) and have led to the wide use of an elicitation procedure known as the discourse completion test (DCT). Justified criticisms have been levelled at the DCT, some labelling it an instrument that limits the capturing of authentic communication, and others making it look almost obsolete. However, there are to date no other sociolinguistic data collection instruments that have as many administrative advantages as the DCT, making it, practically speaking, a resource pragmatics testing and teaching will continue to rely on. Furthermore, a better understanding of communication in such a constructed context may help us gain a better understanding of authentic communication. This study investigates the effect of systematic modification to DCT situational prompts used to elicit requests on the responses of native and non-native speakers of English. Modification included enhancing the situational prompts by adding information on a number of social and contextual variables considered relevant to this study. Results indicate that enhancement did not affect the request strategy or amount of internal modification. However, enhancing situational prompts did produce significantly longer, more elaborated requests in both groups. These findings point to the importance of external modification of speech act production and the need for an instrument that can satisfactorily account for both variation and context. This study has implications for both teaching and testing in interlanguage pragmatics.