The Acquisition of Anaphora by Simple Recurrent Networks

This article applies Simple Recurrent Networks (SRNs; Elman 1991, 1993) to the task of assigning an interpretation to reflexive and pronominal anaphora. This task demands more refined sensitivity to syntactic structure than has been previously explored. Measured quantitatively, SRNs perform quite well. However, the way in which they achieve such performance diverges in key respects from the target grammar: (i) linear N-V-reflexive/pronoun sequences affect the SRN's interpretations, even without a relevant structural relation, yielding errors unlike those made by humans; (ii) the SRN's representations distinguish sentence types, inhibiting structural generalization; (iii) the SRN's knowledge of the conditions on anaphoric dependencies fails to generalize to novel lexical items. These results have important consequences not only for the viability of SRNs as models of language learning but also for the systematicity of generalization in neural networks (Hadley 1994; Marcus 1998).