Computational models accurately simulate human language acquisition.
the verdict
INSUFFICIENT LEANING
refutedsupported
the weight of evidence
5 sources for · 0 against
The literature indicates that computational models can simulate aspects of human language acquisition and provide proofs of concept for learnability, but debates persist regarding discrepancies in learning environments and data scale.
Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. However, the learning environments and biases of current artificial learners and humans diverge in ways that weaken the impact of the evidence obtained from learning simulations. For example, today's most effective neural language models are trained on roughly one thousand times the amount of linguistic data available to a typical child. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans. If an appropriate model successfully acquires some target linguistic knowledge, it can provide a proof of concept that the target is learnable in a hypothesized human learning scenario. Plausible model learners will enable us to carry out experimental manipulations to make causal inferences about variables in the learning environment, and to rigorously test poverty-of-the-stimulus-style claims arguing for innate linguistic knowledge in humans on the basis of speculations about learnability. Comparable experiments will never be possible with human subjects due to practical and ethical considerations, making model learners an indispensable resource. So far, attempts to deprive current models of unfair advantages obtain sub-human results for key grammatical behaviors such as acceptability judgments. But before we can justifiably conclude that language learning requires more prior domain-specific knowledge than current models possess, we must first explore non-linguistic inputs in the form of multimodal stimuli and multi-agent interaction as ways to make our learners more efficient at learning from limited linguistic input.
For example, today’s most effective neural language models are trained on roughly one thousand times the amount of linguistic data available to a typical child. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans. If an appropriate model successfully acquires some target linguistic knowledge, it can provide a proof of concept that the target is learnable in a hypothesized human learning scenario.
While this example is idealized, we believe it is generally far more practical to aim for model learners that lack any unfair advantages over humans, than to try to equip models with the full richness of the hypothesized human learning scenario. Ablation experiments can provide a rigorous test for claims common in the language acquisition literature that the input to the learner lacks key evidence for acquiring certain forms of linguistic knowledge ( Chomsky 1971 ; Legate and Yang 2002 ; Lidz et al. 2003 ; Berwick et al. 2011 ; Rasin and Aravind 2021 ) .
From this perspective, not all models or results are created equal. We recommend a strategy in which relatively impoverished model learners are used to obtain proofs of concept for learnability. We justify this recommendation through theoretical considerations about the conditions under which results generalize from models to humans. 2.1 Generalizing Learnability Results from Models to Humans One goal of the study of language acquisition is to determine the necessary and sufficient conditions for language learning in humans. However, there are some things we cannot easily learn just by observing humans.
If we find that our impoverished models fail, the next course of action is to test whether this is really due to the ablation by enriching the model scenario in innocent ways. This could involve adding to the model scenario sensorimotor input, interaction, and other advantages that humans enjoy. 2.3 Applying Ablations to Debates in Language Acquisition The literature on language acquisition has centered around the necessity and sufficiency of innate advantages and environmental advantages.
Model results are averages over three 100M word miniBERTas reported in Zhang et al. 2021 . Most popular ANNs for NLP have been trained on far more words than a human learner. While this was not the case only a few years ago, this trend has only been increasing. Thus, researchers interested in questions about human language acquisition have already begun to intentionally shift their focus to evaluating models trained on more human-scale datasets (
5.2.1 Misconception 1: A Good Model Learner Must be Unbiased One possible misconception is a model learner must be an unbiased tabula rasa in order to prove some innate bias unnecessary for language acquisition. First, this would be an impossible standard to meet, since all learners have some inductive bias. An inductive bias is just a prior over the hypothesis space, and thus a necessary property of any learner ( Mitchell 1980 ) . Second, we know of no claims that humans are totally unbiased learners.
2020b . Parallelization can make this even faster. Very little hands-on work is required during this training period. Experimentation on artificial learners comes with few ethical restrictions. This is in contrast to experiments on human subjects—and especially infants—which must present minimal risk of harm to the subject. By design, ablations are often very harmful to learning outcomes, meaning we can never do an ablation on L1 acquisition in humans. Aside from experimentation on artificial language learning in humans, the only acquisition ablations we can do on language acquisition is in model learners.
In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , pages 2227–2237. Association for Computational Linguistics, 2018. doi: 10.18653/v1/N18-1202 . URL http://aclweb.org/anthology/N18-1202 . Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language Models as Knowledge Bases?
Large language models (LLMs) have come closest among all models to date to mastering human language, yet opinions about their linguistic and cognitive capabilities remain split. Here, we evaluate LLMs using a distinction between formal linguistic competence (knowledge of linguistic rules and patterns) and functional linguistic competence (understanding and using language in the world). We ground this distinction in human neuroscience, which has shown that formal and functional competence rely on different neural mechanisms. Although LLMs are surprisingly good at formal competence, their performance on functional competence tasks remains spotty and often requires specialized fine-tuning and/or coupling with external modules. We posit that models that use language in human-like ways would need to master both of these competence types, which, in turn, could require the emergence of separate mechanisms specialized for formal versus functional linguistic competence.
ERIC - EJ1286765 - Toward Computational Models of Multilingual Sentence Processing, Language Learning, 2021-Mar Notes FAQ Contact Us Collection Thesaurus Advanced Search Tips Peer reviewed only Full text available on ERIC Collection Thesaurus Browse Thesaurus Include Synonyms Include Dead terms Peer reviewed Direct link ERIC Number: EJ1286765 Record Type: Journal Publication Date: 2021-Mar Pages: 26 Abstractor: As Provided ISBN: N/A ISSN: ISSN-0023-8333 EISSN: N/A Available Date: N/A Toward Computational Models of Multilingual Sentence Processing Frank, Stefan L.
Language Learning , v71 suppl 1 p193-218 Mar 2021 Although computational models can simulate aspects of human sentence processing, research on this topic has remained almost exclusively limited to the single language case. The current review presents an overview of the state of the art in computational cognitive models of sentence processing, and discusses how recent sentence-processing models can be used to study bi- and multilingualism. Recent results from cognitive modeling and computational linguistics suggest that phenomena specific to bilingualism can emerge from systems that have no dedicated components for handling multiple languages.
Hence, accounting for human bi-/multilingualism may not require models that are much more sophisticated than those for the monolingual case. Descriptors: Multilingualism , Language Processing , Computational Linguistics , Psycholinguistics , Sentences , Bilingualism , Models Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030.
Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us Publication Type: Journal Articles; Reports - Evaluative Education Level: N/A Audience: N/A Language: English Sponsor: N/A Authoring Institution: N/A Grant or Contract Numbers: N/A Author Affiliations: N/A Privacy | Copyright | Contact Us | Selection Policy | API | Metrics Journals | Non-Journals | Download | Submit | Multimedia | Widget
How do we recognize words and assign a pronunciation? Computational models provide a formal description of the mechanisms and principles that guide the reading process. I review and evaluate the Interactive-Activation Model (IAM), Dual Route Cascaded (DRC) model, the Parallel Distributed Processing (PDP) model, and the Connectionist Dual Processing (CDP) model, as well as LEX, a variant of the MINERVA model of memory. I evaluate each model’s ability to account for consistency effects, serial effects, syllable effects, and phonological effects. Consistency effects pose a problem for the rule-ba
researchers have been able to simulate language acquisition using neural network models. The structures and uses of language are related to the formation
Psycholinguistics or psychology of language is the study of the interrelation between linguistic factors and other aspects of mind and brain. The discipline is mainly concerned with the psychological and neurobiological mechanisms that enable humans to learn, comprehend, and produce language.
Psycholinguistics is concerned with the cognitive faculties and processes that are necessary to produce th
the behaviorist perspective, whereby all language must be learned by the child; and
the innatist perspective, which believes that the abstract system of language cannot be learned, but that humans possess an innate language faculty or access to what has been called "universal grammar".
The innatist perspective began in 1959 with Noam Chomsky's critical review of B.F. Skinner's Verbal Behavior (1957). This review helped start what has been called the cognitive revolution in psychology. Chomsky posited that humans possess a special, innate ability for language, and that complex syntactic features, such as recursion, are "hard-wired" in the brain. These abilities are thought to be beyond the grasp of even the most intelligent and social non-humans. When Chomsky asserted that children acquiring a language have a vast search space to explore among all possible human grammars, there was no evidence that children received sufficient input to learn all the rules of their language. Hence, there must be some other innate mechanism that endows humans with the ability to learn language. According to the "innateness hypothesis", such a language faculty is what defines human language and makes that faculty different from even the most sophisticated forms of animal communication.
The field of linguistics and psycholinguistics has since been defined by pro-and-con reactions to Chomsky. The view in favor of Chomsky still holds that the
Psycholinguistics or psychology of language is the study of the interrelation between linguistic factors and other aspects of mind and brain. The discipline is mainly concerned with the psychological and neurobiological mechanisms that enable humans to learn, comprehend, and produce language.
Psycholinguistics is concerned with the cognitive faculties and processes that are necessary to produce the grammatical constructions of language. It is also concerned with the perception of these constructions by a listener.
Initial forays into psycholinguistics were in the philosophical and educational fields, mainly due to their location in departments other than applied sciences (e.g., cohesive data on
language development: how do children learn language?
language comprehension: how do people understand language?
language production: how do people speak or sign language?
second language acquisition: how do people who already know one language acquire another one?
A researcher interested in language comprehension may study word recognition during reading, to examine the processes involved in the extraction of orthographic, morphological, phonological, and semantic information from patterns in printed text. A researcher interested in language production might study how words are prepared to be spoken starting from the conceptual or semantic level (this concerns connotation, and possibly can be examined through the conceptual framework concerned with the semantic differential). Developmental psycholinguists study infants' and children's ability to learn and process language.
Psycholinguistics further divide their studies according to the different components that make up human language.
Linguistics-related areas include:
the behaviorist perspective, whereby all language must be learned by the child; and
the innatist perspective, which believes that the abstract system of language cannot be learned, but that humans possess an innate language faculty or access to what has been called "universal grammar".
The innatist perspective began in 1959 with Noam Chomsky's critical review of B.F. Skinner's Verbal Behavior (1957). This review helped start what has been called the cognitive revolution in psychology. Chomsky posited that humans possess a special, innate ability for language, and that complex syntactic features, such as recursion, are "hard-wired" in the brain. These abilities are thought to be beyond the grasp of even the most intelligent and social non-humans. When Chomsky asserted that children acquiring a language have a vast search space to explore among all possible human grammars, there was no evidence that children received sufficient input to learn all the rules of their language. Hence, there must be some other innate mechanism that endows humans with the ability to learn language. According to the "innateness hypothesis", such a language faculty is what defines human language and makes that faculty different from even the most sophisticated forms of animal communication.
The field of linguistics and psycholinguistics has since been defined by pro-and-con reactions to Chomsky. The view in favor of Chomsky still holds that the human ability to use language (specifically the ability to use recursion) is qualitatively different from any sort of animal ability.
The view that language must be learned was especially popular before 1960 and is well represented by the mentalistic theories of Jean Piaget and the empiricist Rudolf Carnap. Likewise, the behaviorist school of psychology puts forth the point of view that language is a behavior shaped by conditioned response; hence it is learned. The view that language can be learned has had a recent resurgence inspired by emergentism. This view challenges the "innate" view as scientifically unfalsifiable; that is to say, it cannot be tested. With the increase in computer technology since the 1980s, researchers have been able to simulate language acquisition using neural network models.
Everything we examined (5)
This check searched the claim as stated. It did not run a separate search for evidence against it.