Course description

This course provides an introduction to deep learning, a methodology to train hierarchical machine learning models. Deep learning is a special form of machine learning where rich data representations are simultaneously learned with the model, thus eliminating the need to engineer features by hand.

The course begins with a comprehensive study of feedforward multilayer neural networks, which are the model of choice for most hierarchical representation learning algorithms. Other models covered in this course include convolutional neural networks, autoencoders, recurrent neural networks and long short-term memory models. Several lectures are devoted to discuss practical strategies to improve the performance of deep models, such as regularization, data augmentation, pre-training, dropout, multi-task learning and advanced optimization methods. The course also reviews modern applications of deep learning, such as image categorization, speech recognition, natural language processing, and audio analysis.

The course is aimed at undergraduate and graduate students who have taken COSC 74/174, Machine Learning and Statistical Data Analysis.

Administrative information

Instructor

Lorenzo Torresani

Teaching assistants

Yiren Jian

Lectures

MWF 10:10am-11:15pm | x-hour (used occasionally to make up cancelled classes) Th 12:15pm-1:05pm
Kemeny 008

Software

For the homework assignments we will be using MatConvNet, a MATLAB toolbox for simple and efficient implementation of deep learning methods. Dartmouth College provides free MATLAB licenses to students. You can download and install Matlab on your personal computer by following the instructions listed here.

Textbook

There is no required textbook for the class. Slides will be made available to students. The following online book by Ian Goodfellow, Yoshua Bengio and Aaron Courville can be used as additional reference: http://www.deeplearningbook.org.

Getting assistance

The platforms for receiving assistance in CS78/178 are exclusively Piazza and office hours. Do not send us emails. If you have a question that needs to be discussed one-on-one with the instructor, then come to his office hours or talk to him at the end of class.

Piazza

If you have not already done so, sign up for a Piazza account on Canvas. We will use Piazza extensively and this should be the first place to post questions about the course materials, homework assignments, or general topics on deep learning. Please check Piazza regularly. (We recommend that you set your preferences to receive daily Piazza email digests or immediate email notifications.) You must not post code or other specific information about your homework solutions. Similarly, do not post links to external references. You are not even allowed to search online for auxiliary data, model or software resources. The homework assignments are self-contained and must be solved only with the data and references provided in our homework documentation, or the materials discussed in class.

Office Hours

You may seek guidance in person during the following office hours
- Tuesday, 3pm-4pm, Sudikoff 121: Yiren
- Thursday, 12:15pm-1:15pm, Sudikoff 121: Yiren
- Friday, 3pm-5pm, Sudikoff 120: Lorenzo

Take full advantage of office hours and of communications via Piazza. At the same time keep in mind that we will not be able to accommodate requests for additional support or individual meetings.

Grading and policies

Grading

Grades will be based on four homework assignments and a final exam. The homework assignments will be worth 60% of the course grade and the final exam will be worth 40%. The homework assignments will require implementation of several deep learning models applied to various domains. Grades will be based on the ability to reproduce the performance of specified baseline models. Bonus points will be given for further improving the accuracy of the baseline models on test benchmarks, for which labels will not be released (we will provide a script to evaluate your models on the test sets a limited number of times). Homework assignments must be done individually. Sharing code or model specifications is strictly prohibited. Homework discussions are allowed only on Piazza, according to the policy outlined above. Models implemented for the homework assignments must be trained exclusively on the data provided with the assignment. The improvements made to the baseline models must be your own. Remember: you cannot search for data, architectures or reference models on the Web to solve the homework assignments.

Late homework submissions

Each student has 4 free late days to be used over the course of the term as he/she likes. Once these days are used up, any homework turned in late will be penalized 25% per late day. No exception! Any portion of a late day is counted as one full day. Assignments are typically due at 11:59 pm of the due date. Your code and models must be be submitted electronically via Canvas.

Homework and Exam Schedule

Homework 1: out on 1/9/2019; due on 1/23/2019.

Homework 2: out on 1/23/2019; due on 2/6/2019.

Homework 3: out on 2/6/2019; due on 2/20/2019.

Homework 4: out on 2/20/2019; due on 3/6/2019.

Final exam: 3/9/2019 3pm-5pm.

No-laptop policy

We have a no-laptop policy in class (typing, texting or engaging in activities unrelated to the lecture is also not permitted). This policy will be strictly enforced in order to encourage active participation by all students and to avoid distracting people that are focusing on the lecture. Class attendance is highly recommended and considered necessary to do well in the course. However, it is not mandatory. If you come to class you are expected to obey this policy. A penalty of 5% will be applied to the final grade every time this policy is violated.

Auditing

Please contact the instructor if you want to audit the class.

Academic integrity

As already outlined above, homework discussion with other students is only allowed via the medium of Piazza, according to the policy outlined above. You are not allowed to search online for auxiliary software, reference models, or additional data to solve the homework assignments. Your submission must be entirely your own work. That is, the code and the answers that you submit must be created, typed, and documented by you alone, based exclusively on the materials discussed in class and released with the homework assignments.

You cannot collaborate or copy in any way during the exams. The exams will be will be closed-book, closed-notes, closed laptop.

These rules will be strictly enforced and any violation will be treated seriously.