Today’s large language models generate coherent, grammatical text. This makes it easy, perhaps too easy, to see them as “thinking machines”, capable of performing tasks that require abstract knowledge and reasoning. I will draw a distinction between formal competence (knowledge of linguistic rules and patterns) and functional competence (understanding and using language in the world). Language models have made huge progress in formal linguistic competence, with important implications for linguistic theory. Even though they remain interestingly uneven at functional linguistic tasks, they can distinguish between grammatical and ungrammatical sentences in English, and between possible and impossible languages. As such, language models can be an important tool for linguistic theorizing. In making this argument, I will draw on a study of language models and constructions, specifically the A+Adjective+Numeral+Noun construction (“a beautiful five days in Montreal”). In a series of experiments small language models are treined on human-scale corpora, systematically manipulating the input corpus and pretraining models from scratch. I will discuss implications of these experiments for human language learning.

 

References

K. Mahowald, A. Ivanova, I. Blank, N. Kanwisher, J. Tenenbaum, E. Fedorenko. 2024. Dissociating Language and Thought in Large Language Models: A Cognitive Perspective. Trends in Cognitive Sciences.

K. Misra, K. Mahowald. 2024. Language Models Learn Rare Phenomena From Less Rare Phenomena: The Case of the Missing AANNs. Preprint.

J. Kallini, I. Papadimitriou, R. Futrell, K. Mahowald, C. Potts. 2024. Mission: Impossible Language Models. Preprint.

J. Hu, K. Mahowald, G. Lupyan, A. Ivanova, R. Levy. 2024. Language models align with human judgments on key grammatical constructions. Preprint.

H. Lederman, K. Mahowald. 2024. Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs. Preprint.

K. Mahowald. 2023. A Discerning Several Thousand Judgments: GPT-3 Rates the Article Adjective + Numeral + Noun Construction. Proceedings of EACL 2023.