Language acquisition in the absence of explicit negative evidence: can simple recurrent networks obviate the need for domain-specific learning devices?
Explore the source record for details and available documents.
Concept Formation↗
Biomedical subjects
G F Marcus
Publications and source records attributed to G F Marcus.
Rule learning by seven-month-old infants.
A fundamental task of language acquisition is to extract abstract algebraic rules. Three experiments show that 7-month-old infants attend longer to sentences with unfamiliar structures than to sentences with familiar structures. The design of the artificial language task used in these experiments ensured that this discrimination could not be performed by counting, by a system that is sensitive only to transitional probabilities, or by a popular class of simple neural network models. Instead, these results suggest that infants can represent, extract, and generalize abstract algebraic rules.
Brain↗
Rethinking eliminative connectionism.
Humans routinely generalize universal relationships to unfamiliar instances. If we are told "if glork then frum," and "glork," we can infer "frum"; any name that serves as the subject of a sentence can appear as the object of a sentence. These universals are pervasive in language and reasoning. One account of how they are generalized holds that humans possess mechanisms that manipulate symbols and variables; an alternative account holds that symbol-manipulation can be eliminated from scientific theories in favor of descriptions couched in terms of networks of interconnected nodes. Can these "eliminative" connectionist models offer a genuine alternative? This article shows that eliminative connectionist models cannot account for how we extend universals to arbitrary items. The argument runs as follows. First, if these models, as currently conceived, were to extend universals to arbitrary instances, they would have to generalize outside the space of training examples. Next, it is shown that the class of eliminative connectionist models that is currently popular cannot learn to extend universals outside the training space. This limitation might be avoided through the use of an architecture that implements symbol manipulation.
Humans↗
Can connectionism save constructivism?
Constructivism is the Piagetian notion that learning leads the child to develop new types of representations. For example, on the Piagetian view, a child is born without knowing that objects persist in time even when they are occluded; through a process of learning, the child comes to know that objects persist in time. The trouble with this view has always been the lack of a concrete, computational account of how a learning mechanism could lead to such a change. Recently, however, in a book entitled Rethinking Innateness. Elman et al. (Elman, J.L., Bates, E., Johnson, M.H., Karmiloff-Smith, A., Parisi, D., Plunkett, K., 1996. Rethinking Innateness: A Connectionist Perspective on Development. Cambridge, MA: MIT Press) have claimed that connectionist models might provide an account of the development of new kinds of representations that would not depend on the existence of innate representations. I show that the models described in Rethinking Innateness depend on innately assumed representations and that they do not offer a genuine alternative to nativism. Moreover, I present simulation results which show that these models are incapable of deriving genuine abstract representations that are not presupposed. I then give a formal account of why the models fail to generalize in the ways that humans do. Thus, connectionism, at least in its current form, does not provide any support for constructivism. I conclude by sketching a possible alternative.
Child Development↗
German inflection: the exception that proves the rule.
Language is often explained as the product of generative rules and a memorized lexicon. For example, most English verbs take a regular past tense suffix (ask-asked), which is applied to new verbs (faxed, wugged), suggesting the mental rule "add -ed to a Verb." Irregular verbs (break-broke, go-went) would be listed in memory. Alternatively, a pattern associator memory (such as a connectionist network) might record all past tense forms and generalize to new ones by similarity; irregular and regular patterns would differ only because of their different numbers of verbs. We present evidence that mental rules are indispensible. A rule concatenates a suffix to a symbol for verbs, so it does not require access to memorized verbs or their sound patterns, but applies as the "default," whenever memory access fails. We find 21 such circumstances for regular past tense formation, including novel, unusual-sounding, and rootless and headless derived words; in every case, people inflect them regularly (explaining quirks like flied out, sabre-tooths, walkmans). Contrary to the connectionist account, these effects are not due to regular words constituting a large majority of vocabulary. The German participle -t applies to a much smaller percentage of verbs than its English counterpart, and the German plural -s applies to a small minority of nouns. But the affixes behave in the language like their English counterparts, as defaults. We corroborate this effect in two experiments eliciting ratings of participle and plural forms of novel German words. Thus default suffixation is not due to numerous regular words reinforcing a pattern in associative memory. Because default cases do not occupy a cohesive similarity space, but do correspond to the range of a symbol, they are evidence for a memory-independent, symbol-concatenating mental operation.
Adult↗
The acquisition of the English past tense in children and multilayered connectionist networks.
The apparent very close similarity between the learning of the past tense by Adam and the Plunkett and Marchman model is exaggerated by several misleading comparisons--including arbitrary, unexplained changes in how graphs were plotted. The model's development differs from Adam's in three important ways: Children show a U-shaped sequence of development which does not depend on abrupt changes in input; U-shaped development in the simulation occurs only after an abrupt change in training regimen. Children overregularize vowel-change verbs more than no-change verbs; the simulation overregularizes vowel-change verbs less often than no-change verbs. Children, including Adam, overregularize more than they irregularize; the simulation overregularized less than it irregularized. Interestingly, the RM model--widely criticized as being inadequate--does somewhat better, correctly overregularizing vowel-change verbs more often than no-change verbs, and overregularizing more often than it irregularizes. Although Plunkett and Marchman's (1993) state of the art model incorporated hidden layers and back-propagation, used a more realistic phonological coding scheme, and explored a broader range of parameters than Rumelhart and McClelland's model, their results are farther from psychological reality. It is unknown whether any connectionist model can mimic a child's performance without resorting to unrealistic exogenous changes in the training or input, but it is clear that adding a hidden-layer and back-propagation does not ensure a solution.
Child Language↗
Children's overregularization of English plurals: a quantitative analysis.
This paper brings a quantitative study of children's noun plural overregularizations (foots, mans) to bear on recent comparisons of connectionist and symbolic models of language. The speech of 10 English-speaking children (aged 1;3 to 5;2) from the CHILDES database (MacWhinney & Snow, 1985, 1990) were analysed. The rate of noun overregularization is low, mean = 8.5%, demonstrating that children prefer correct to overregularized forms. Rates of noun overregularization are not significantly different from their rates of past tense overregularization, and noun plurals, like verb past tenses, follow a U-shaped developmental curve in which correct irregulars precede the first overregularized forms. These facts suggest that plural and past tense overregularizations are caused by similar underlying processes. The results pose challenges to connectionist models, but are consistent with Marcus et al.'s (1992) blocking-and-retrieval-failure model in which regulars are generated by a default rule while irregulars are retrieved from the lexicon.
Child Language↗
Sensitivity of children's inflection to grammatical structure.
What is the input to the mental system that computes inflected forms like walked, came, dogs, and men? Recent connectionist models feed a word's phonological features into a single network, allowing it to generalize both regular and irregular phonological patterns, like stop-stopped, step-stepped and fling-flung, cling-clung. But for adults, phonological input is insufficient: verbs derived from nouns like ring the city always have regular past tense forms (ringed), even if they are phonologically identical to irregular verbs (ring the bell). Similarly, nouns based on names, like two Mickey Mouses, and compounds based on possessing rather than being their root morpheme, such as two sabertooths, take regular plurals, even when they are homophonous with irregular nouns like mice and teeth. In four experiments, testing 70 three- to ten-year-old children, we found that children are sensitive to such nonphonological information: they were more likely to produce regular inflected forms for forms like to ring ('to put a ring on') and snaggletooth (a kind of animal doll with big teeth) than for their homophonous irregular counterparts, even when these counterparts were also extended in meaning. Children's inflectional systems thus seem to be like adults': irregular forms are tied to the lexicon but regular forms are computed by a default rule, and words are represented as morphological tree structures reflecting their derivation from basic word roots. Such structures, which determine how novel complex words are derived and interpreted, also govern whether words with irregular sound patterns will be regularized: a word can be irregular only if its structure contains an irregular root in 'head' position, allowing the lexically stored irregular information to percolate up to apply to the word as a whole. In all other cases, the inflected form is compouted by a default regular rule. This proposal fits the facts better than alternatives appealing to ambiguity reduction or semantic similarity to a word's central sense. The results, together with an analysis of adult speech to children, suggest that morphological structure and a distinction between mechanisms for regular and irregular inflection may be inherent to the design of children's language systems.
Child↗
Negative evidence in language acquisition.
Whether children require "negative evidence" (i.e., information about which strings of words are not grammatical sentences) to eliminate their ungrammatical utterances is a central question in language acquisition because, lacking negative evidence, a child would require internal mechanisms to unlearn grammatical errors. Several recent studies argue that parents provide noisy feedback, that is, certain discourse patterns that differ in frequency depending on the grammaticality of children's utterances. However, no one has explicitly discussed how children could use noisy feedback, and I show that noisy feedback is unlikely to be necessary for language learning because (a) if noisy feedback exists it is too weak: a child would have to repeat a given sentence verbatim at least 85 times to decide with reasonable certainty that it is ungrammatical; (b) no kind of noisy feedback is provided to all children at all ages for all types of errors; and (c) noisy feedback may be an artifact of defining parental reply categories relative to the child's utterance. For example, because nearly all parental speech is grammatical, exact repetitions (verbatim repetitions of child utterances) necessarily follow more of children's grammatical utterances than their ungrammatical utterances. There is no evidence that noisy feedback is required for language learning, or even that noisy feedback exists. Thus internal mechanisms are necessary to account for the unlearning of ungrammatical utterances.
Child Language↗
Regular and irregular inflection in the acquisition of German noun plurals.
In this paper we study the acquisition of German noun plurals in relation to the question of how children represent regular and irregular inflection. Pinker and Prince (1992) have demonstrated several dissociations between regular and irregular inflection in the English past tense system. However, in English, the default status of -ed is confounded with its high frequency; therefore inflectional systems other than English past tense formation must be examined. The noun plural system in German is particularly interesting, because most nouns have irregular plurals in German and the regular (default) plural is less frequent than several of the irregular plurals. Thus it is unclear how a language learner determines whether German even has a regular plural, and if so what form it takes. Based on longitudinal data from impaired and unimpaired monolingual German-speaking children, we find a striking, statistically significant correlation: plural affixes that are used in overregularizations, namely -n or -s, are left out within compounds. This correlation shows that even impaired children are sensitive to the distinction between regular and irregular morphology. We propose a linguistic analysis of the correlation in terms of Kiparsky's (1982, 1985) level-ordering model plus an additional ordering condition on affixes: default (regular) affixes cannot serve as input to compounding processes.
Child↗
Overregularization in language acquisition.
Children extend regular grammatical patterns to irregular words, resulting in overregularizations like comed, often after a period of correct performance ("U-shaped development"). The errors seem paradigmatic of rule use, hence bear on central issues in the psychology of rules: how creative rule application interacts with memorized exceptions in development, how overgeneral rules are unlearned in the absence of parental feedback, and whether cognitive processes involve explicit rules or parallel distributed processing (connectionist) networks. We remedy the lack of quantitative data on overregularization by analyzing 11,521 irregular past tense utterances in the spontaneous speech of 83 children. Our findings are as follows. (1) Overregularization errors are relatively rare (median 2.5% of irregular past tense forms), suggesting that there is no qualitative defect in children's grammars that must be unlearned. (2) Overregularization occurs at a roughly constant low rate from the 2s into the school-age years, affecting most irregular verbs. (3) Although overregularization errors never predominate, one aspect of their purported U-shaped development was confirmed quantitatively: an extended period of correct performance precedes the first error. (4) Overregularization does not correlate with increases in the number or proportion of regular verbs in parental speech, children's speech, or children's vocabularies. Thus, the traditional account in which memory operates before rules cannot be replaced by a connectionist alternative in which a single network displays rotelike or rulelike behavior in response to changes in input statistics. (5) Overregularizations first appear when children begin to mark regular verbs for tense reliably (i.e., when they stop saying Yesterday I walk). (6) The more often a parent uses an irregular form, the less often the child overregularizes it. (7) Verbs are protected from overregularization by similar-sounding irregulars, but they are not attracted to overregularization by similar-sounding regulars, suggesting that irregular patterns are stored in an associative memory with connectionist properties, but that regulars are not. We propose a simple explanation. Children, like adults, mark tense using memory (for irregulars) and an affixation rule that can generate a regular past tense form for any verb. Retrieval of an irregular blocks the rule, but children's memory traces are not strong enough to guarantee perfect retrieval. When retrieval fails, the rule is applied, and overregularization results.
Age Factors↗