CS078/CS278, Winter 2021
Deep Learning
About Schedule

This page contains a tentative draft of the syllabus. It will be updated frequently with current and upcoming topics. Chapter references, when available, are to the recommended course textbook, Deep Learning, by Ian Goodfellow, Yoshua Bengio and Aaron Courville.

DateTopicsReferencesOutDue
January 7Course introduction; logistic regression = 1 neuronSec. 5.5, 5.7.1
January 8 (x-hour)Tutorial on PyTorch (part 1)
January 12From single neuron to multilayer neural networksSec. 6.1-6.4HW1
January 14Backpropagation (part 1)Sec. 6.5
January 15 (x-hour)Tutorial on PyTorch (part 2)
January 19Backpropagation (part 2)
January 21Strategies for training deep models (preprocessing,
regularization, initialization, hyper-parameters)		Sec. 7.1, 8.3, 8.4, 11.4, 11.5
January 22 (x-hour)
January 26Convolutional neural networks (motivation, local
connectivity, parameter sharing)		Sec. 9.1, 9.2HW2HW1
January 28Convolutional neural networks
(striding, pooling)		Sec. 9.3
January 29 (x-hour)
February 2Practical tricks for CNNs
(filter size, depth, study of popular architectures)		Sec. 9.4
February 4Data scarcity:
data augmentation, transfer learning and fine tuning		DeCAF paper
February 5 (x-hour)
February 9Dense prediction: fully-convolutional networks,
transposed convolution, skip connections
FCN paperHW3HW2
February 11Midterm exam
February 12 (x-hour)
February 16Recurrent neural networksSec. 10.1, 10.2, 10.4, 10.5
February 18LSTMsSec. 10.7, 10.10, 10.11
February 19 (x-hour)
February 23Advanced optimization:
Nesterov momentum, Adagrad, RMSProp		Sec. 8.5
HW4HW3
February 25Batch normalization, dropout,
residual learning		Sec. 7.12, BN paper,
dropout paper, ResNet paper
February 26 (x-hour)
March 2Networks for video understanding
March 4Unsupervised learning: undercomplete and denoising
autoencoders, unsupervised pretraining		Sec. 14.1, 14.5, 14.9, 15.1
March 5 (x-hour)
March 9Review of topics for final exam
HW4
TBDFinal exam