The science and stakes of Large Language Models

We are delighted to announce that the ISC 2024 Summer School - Chatting Minds: The science and stakes of Large Language Models will be held in hybrid mode, both online and in person, from June 3rd to June 14, 2024.  

_________

UQAM's Institut des Sciences Cognitives has been organizing summer schools biennially since 2003, on themes such as categorization, social cognition, the origin of language, the evolution and function of consciousness, web and mind, reasoning, animal sentience and cognition, and the cognitive challenges of cognitive change.

For summer 2024, the Institute of Cognitive Sciences intends to address the theme of large language models (LLMs). LLMs are artificial intelligence systems based on deep learning and trained on massive linguistic datasets taken from large corpora of varied texts from the Internet and other sources. They are designed to manipulate natural languages in a way that enables them to converse naturally with human beings, answering their questions or receiving instructions from them. With the many foreseeable improvements in technology, their current capabilities will only improve and expand. Since the public deployment of these systems is quite recent (October 2022 for the first ChatGPT), their media coverage has been intense, and they raise important scientific, ethical and social questions. We believe that a summer school will be of interest to students and researchers from many fields here at UQÀM as well as around the world.

The summer school will examine major language models from four perspectives (some more profound than others): 

1) The technical basis on which LLMs are developed 

2) What LLMs teach us about human cognition and language

3) What ethical issues LLMs raise, and finally 

4) What are the educational, economic, social and political consequences of their large-scale deployment. 

The school will consist of a series of forty 45-minute lectures, each followed by 15-minute discussion periods, ten 60-minute plenary sessions (one per day), a poster session on the topics covered by summer school and, for registered students, a paper evaluated by a member of the institute's scientific committee.

To consult our list of lecturers, who are all leaders in their respective fields, head to the "Speakers" tab at the top of the page!

Throughout the next two weeks we will have, on lunchtime, hands-on workshops directly linked to LLMs. These workshops are free - here are the topics covered: 

Basic tools 

First steps with HuggingFace 

1. Loading datasets
2. Loading models from hub
3. Loading models locally
4. Loading models for specific tasks
5. Saving models locally

Using Google Colab 

Using the SentenceTransformers library NLP 

Fine-Tuning 1: 

1. General idea
2. Sequence classification

Fine-Tuning 2: 

1. Token classification
2. NLI

Pre-training, domain adaptation: 

1. Pretraining blank models
2. Domain adaptation of pre-trained models

Prompt engineering 

Topic modelling

Debiasing word embeddings

Building a Transformer from scratch 

Use in linguistics, cognitive science

1. Semantic probing of LLMs
2. Using LLMS as a proxy for human language to conduct linguistics studies