A Hands-on Introduction to Natural Language Processing (NLP)

About this course

This course was created by Prof. Mohammad Ghassemi in Fall of 2020 as part of the CSE 842 class at Michigan State University. The course provides a step-by-step guide to NLP and makes no assumptions that you have a background in the material (NLP or Machine Learning). The content in this repository will teach you:

1. How to collect and process text data.
2. How to generate text using language models.
3. How to classify text using machine learning.
4. How to use and tune state-of-the-art sequence-to-sequence models, including transformers.
5. How to process speech signals.

All lectures are hosted on Youtube and can be consumed at your own pace (see links below). At the end of (most) every lecture there is a tutorial + homework assignment that will demonstrate how to perform NLP tasks in Python. The Python Notebooks are available through the links below, and in the Homework folder.

Introduction

• Lectures:
  • a. Introduction to the Course
  • b. Overview of Syllabus
  • c. Overview of Course Tools
  • d. Overview of the Course Project
• HW0: Setting up your notebook and Gitlab Repo
• Project: Guidelines

NLP Fundamentals and N-gram Language Models

• Optional Readings:
  • Chapter 2 and Chapter 3
• Lecture:
  • Introduction
  • a. Language and its properties
  • b. Regular Expressions
  • c. Text Normalization
  • d. Edit Distance
  • e. N-gram Language Models
  • f. Generalization and Smoothing
  • HW1 Tutorial
• HW1 and Code Tutorial: Basic data manipulations, representations and statistics

Niave Bayes, Sentiment Classification, Logistic Regression

• Optional Readings:
  • Chapter 4 and Chapter 5
• Lecture:
  • Introduction
  • Overview
  • a. Numerical Representations of Text
  • b. Naive Bayes
  • c. Logistic Regression
  • d. Performance Metrics
  • HW2 Tutorial
• HW2 and Code Tutorial: Supervised language classification models and their assessment

Vector Semantics, Embeddings, Neural Language Models

• Optional Readings
  • Chapter 6 and Chapter 7
  • PCA
  • Word2Vec
• Lecture:
  • Introduction
  • Overview
  • a. Simple Contextual Representations
  • b. Gradient Descent
  • c. Neural Networks
  • d. Neural Langauge Models
  • HW3 Tutorial
• HW3 and Code Tutorial: Embeddings and Neural Networks

Modeling Text as a Sequence

• Optional Readings
  • Chapter 8 and Chapter 9
• Lecture:
  • Overview
  • Introduction
  • a. Hidden Markov Models
  • b. Maximum Entropy Markov Models
  • c. Recurrent Neural Networks
  • HW4 Tutorial
• HW4 and Code Tutorial: Sequence Models

Encoder-Decoder Models, Attention and Transformers

• Optional Readings
  • Chapter 10, Viswani, 2017
• Lecture:
  • Overview
  • Introduction
  • a. Encoder-Decoder Networks
  • b. Attention
  • c. Transformers
  • HW5 Tutorial
• HW5 and Code Tutorial: Transformers

Constituencies, Parsing and Dependency

• Optional Readings
  • Chapter 12, Chapter 13, Chapter 14, Chapter 15
• Lecture:
  • Overview
  • Introduction
  • a. Context Free Grammar
  • b. Constituency Parsing
  • c. Statistical Consituency Parsing
  • d. Dependency Parsing
• HW6 and Code Tutorial: Context free grammar

Speech Processing

• Optional Readings
  • Chapter 27
• Lecture:
  • Overview
  • Introduction
  • a. Phonetics
  • b. Speech Signals
  • HW7 Tutorial
• HW7 and Code Tutorial: Speech Analysis