This page contains a tentative draft of the syllabus. It will be updated frequently with current and upcoming topics. Chapter references, when available, are to the recommended course textbook, Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
| Date | Topics | References | Out | Due |
|---|---|---|---|---|
| January 4 | Course introduction | |||
| January 7 | Logistic regression as a 1-neuron network | Sec. 5.5, 5.7.1 | ||
| January 9 | From single neuron to multilayer neural networks | Sec. 6.1-6.4 | HW1 | |
| January 10 (x-hour) | Tutorial on MatConvNet (Yiren) | |||
| January 11 | Backpropagation (part 1) | Sec. 6.5 | ||
| January 14 | Backpropagation (part 2) | |||
| January 16 | Practical strategies for training deep models (preprocessing,regularization) | Sec. 7.1, 8.3 | ||
| January 18 | More strategies for training deep models (initialization, hyper-parameters, debugging) | Sec. 8.4, 11.4, 11.5 | ||
| January 23 | Convolutional neural networks (motivation, local connectivity, parameter sharing) | Sec. 9.1, 9.2 | HW2 | HW1 |
| January 25 | Convolutional neural networks (striding, pooling) | Sec. 9.3 | ||
| January 28 | Practical tricks for CNNs (filter size, depth) | Sec. 9.4 | ||
| January 30 | Practical tricks for CNNs (study of popular architectures) | |||
| February 1 | Data scarcity: data augmentation, transfer learning and fine tuning | DeCAF paper | ||
| February 4 | Dense prediction: fully-convolutional networks (part 1) | FCN paper | ||
| February 6 | Dense prediction: fully-convolutional networks (part 2) | HW3 | HW2 | |
| February 8 | Dense prediction: transposed convolution, skip connections | |||
| February 11 | Recurrent neural networks (part 1) | Sec. 10.1, 10.2 | ||
| February 13 | Recurrent neural networks (part 2) | Sec. 10.4, 10.5 | ||
| February 15 | LSTMs | Sec. 10.7, 10.10, 10.11 | ||
| February 18 | Advanced optimization: batch normalization, dropout | Sec. 7.12 BN paper, dropout paper | ||
| February 20 | Advanced optimization: Nesterov momentum, Adagrad, RMSProp | Sec. 8.5 | HW4 | HW3 |
| February 22 | Unsupervised learning: undercomplete and denoising autoencoders | Sec. 14.1, 14.5, 14.9 | ||
| February 25 | Unsupervised learning: unsupervised pretraining | Sec. 15.1 | ||
| February 27 | Self-supervised learning (part 1) | |||
| March 1 | Self-supervised learning (part 2) | |||
| March 4 | Q&A, review session | HW4 | ||
| March 9 (3pm-5pm) | Final exam |