Course Content

Read an outline of topics and bibliography for this course

Course Outline

Class 1: "Foundations of NLP and LLMs"

• Content: An introductory overview of NLP methods, including key concepts and comparisons with other NLP techniques. Discussion of ethical concerns.

Class 2: "Transformers, BERT, and Large Language Models"

• Content: A deep dive into transformer architecture, with a particular focus on BERT and its practical applications in NLP. Explore the fundamentals and practical roles of Large Language Models (LLMs).

Class 3: "Few-Shot Learning and Hugging Face Tools"

• Content: Learn how to efficiently apply few-shot learning techniques for fast problem-solving in NLP. Gain proficiency in using Hugging Face's tools, libraries, and APIs for tasks related to Large Language Models (LLMs).

Class 4: "Prompt Engineering for LLMs"

• Content: Develop expertise in crafting precise prompts that effectively extract desired information from Large Language Models (LLMs).

Class 5: "Fine-Tuning LLMs and Adaptation"

• Content: Master the art of customizing pre-trained Large Language Models (LLMs) for specific tasks and domains. Learn how to adapt these models to your unique data and requirements.

References

• Vaswani et al., (2017), Attention Is All You Need
• Devlin et al., (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Alamar, Jay, (2018) The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
• Alamar, Jay, (2018) The Illustrated Transformer
• Wolf et al. (2019) HuggingFace's Transformers: State-of-the-art Natural Language Processing
• ​​Sun et al., (2019), How to Fine-Tune BERT for Text Classification?
• Brown et al., (2020), Language Models are Few-Shot Learners
• Gao, Tianyu, (2021), Prompting: Better Ways of Using Language Models for NLP Tasks
• Timo Schick and Hinrich Schütze (2021). Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference
• Bender et al., (2021), On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 
• Strubell et al. (2019) Energy and Policy Considerations for Deep Learning in NLP
• Dodge et al. (2022) Measuring the Carbon Intensity of AI in Cloud Instances
• Sheng et al. (2019) The Woman Worked as a Babysitter: On Biases in Language Generation
   

Prerequisites

Competence in Python, R, Jupyter Notebooks, and matrix algebra is recommended for optimal course comprehension. Participants with limited or no experience with programming in Python and R are encouraged to register to the "Introduction to Python and R Programming for Data Science" course, scheduled for March 5 and 6.

While a graduate-level background in Statistics, Machine Learning, or Data Science is not mandatory to attend the course, it is highly desirable. Participants with limited experience in these fields are encouraged to register for the "Foundations of Data Science" course, scheduled for March 6, 7 and 8.

Students are required to have their own laptop or desktop computer and a stable Internet connection to fully engage in and benefit from the course.