Language Technology I

A Graduate Course

Info

Instructors: Dr. Jon Dehdari and Dr. Asad Sayeed
Class Location: (Former) CiP Room, building C7.2
Class Times: Lecture: Mondays 14:00-16:00 (c.t); Lab: Wednesdays 16:00-18:00 (s.t)
Class Dates: Oct. 31st - Feb. 15th
Jon's Offices: either room 1.15, building A2.2, or room 1.11 building D3.1
Asad's Office: room 3.04, building C7.4
Final exam: Feb. 22nd, 16:00 - 18:00, Conference Room of Building C7.4

Purpose

Language Technologies I teaches the theoretical foundation of modern computational linguistics and natural language processing. This includes important machine learning techniques.

Outline

1. Formal models of language: possibilities (homework)
2. Statistical models of language: probabilities (homework)
3. Applications of language models
4. n-gram language models and smoothing (more info) (homework) (training data) (testing data) (example transcript) (Bad-Turing transcript) (lazy tokenization script)
5. Parts of speech, word clusters, and class-based language models
6. Log-linear models (homework)
7. Word vectors, and applications (homework)
8. Feedforward neural networks and autoencoders (homework)
9. Recurrent neural networks and their language models (homework) (example transcript)
10. Probabilistic context-free grammars, parsing, and syntactic language models (homework - small updates)
11. Sequence-to-sequence models and neural machine translation
12. Convolutional networks and character-based models of language

External Links

• Neural Network Software:

  • Keras, an easy neural network library. You can install it by typing the following from the command line:

    pip3 install --user keras

  • List of other software for neural networks

  • Another list of recurrent neural network stuff

• Free Corpora:
  • WMT Data, both parallel and monolingual corpora
  • ACL Wiki, "Resources by Language"
  • OPUS - open parallel corpora
• Corpus Processing Tools:
  • https://github.com/jonsafari/habeas-corpus
  • https://github.com/kpu/preprocess
  • http://corpus.tools