Skip to main page content

From Word Models to World Models: Natural Language to the Probabilistic Language of Thought

My Session Status

What:
Talk
Part of:
When:
1:30 PM, Thursday 6 Jun 2024 EDT (1 hour 30 minutes)
Theme:
Large Language Models & Understanding
How do humans make meaning from language? And how can we build machines that think in more human-like ways? "Rational Meaning Construction" combines neural language models with probabilistic models for rational inference. Linguistic meaning is a context-sensitive mapping from natural language into a probabilistic language of thought (PLoT), a general-purpose symbolic substrate for generative world modelling. Thinking can be modelled with probabilistic programs, an expressive representation for commonsense reasoning. Meaning construction can be modelled with large language models (LLMs) that support translation from natural language utterances to code expressions in a probabilistic programming language. LLMs can generate context-sensitive translations that capture linguistic meanings in (1)probabilistic reasoning, (2) logical and relational reasoning, (3) visual and physical reasoning, and (4) social reasoning. Bayesian inference with the generated programs supports coherent and robust commonsense reasoning. Cognitively motivated symbolic modules (physics simulators, graphics engines, and planning algorithms)  provide a unified commonsense-thinking interface from language. Language can drive the construction of world models themselves. We hope this work will lead to cognitive models and AI systems combining the insights of classical and modern computational perspectives.

 

References

Wong, L., Grand, G., Lew, A. K., Goodman, N. D., Mansinghka, V. K., Andreas, J., & Tenenbaum, J. B. (2023). From word models to world models: Translating from natural language to the probabilistic language of thought. arXiv preprint arXiv:2306.12672.

Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2023). Dissociating language and thought in large language models: a cognitive perspective. arXiv preprint arXiv:2301.06627.

Ying, L., Zhi-Xuan, T., Wong, L., Mansinghka, V., & Tenenbaum, J. (2024). Grounding Language about Belief in a Bayesian Theory-of-Mind. arXiv preprint arXiv:2402.10416.

Hsu, J., Mao, J., Tenenbaum, J., & Wu, J. (2024). What’s Left? Concept Grounding with Logic-Enhanced Foundation Models. Advances in Neural Information Processing Systems, 36.

________________

My Session Status

Session detail
Allows attendees to send short textual feedback to the organizer for a session. This is only sent to the organizer and not the speakers.
To respect data privacy rules, this option only displays profiles of attendees who have chosen to share their profile information publicly.

Changes here will affect all session detail pages