CSE 599G1: Deep Learning System
CSE 599W

    
Course Information

Over the past few years, deep learning has become an important technique to successfully solve problems in many different fields, such as vision, NLP, robotics. An important ingredient that is driving this success is the development of deep learning systems that efficiently support the task of learning and inference of complicated models using many devices and possibly using distributed resources. The study of how to build and optimize these deep learning systems is now an active area of research and commercialization, and yet there isn’t a course that covers this topic.

This course is designed to fill this gap.  We will be covering various aspects of deep learning systems, including: basics of deep learning, programming models for expressing machine learning models, automatic differentiation, memory optimization, scheduling, distributed learning, hardware acceleration, domain specific languages, and model serving. Many of these topics intersect with existing research directions in databases, systems and networking, architecture and programming languages. The goal is to offer a comprehensive picture on how deep learning systems works, discuss and execute on possible research opportunities, and build open-source software that will have broad appeal.

We will have two classes per week. The class will either be lecture or  lab/discuss session.
Each lecture will study a specific aspect of deep learning systems.
The lab/discussion session will contain tutorials to implement that specific aspect
and will include case studies of existing systems, such as Tensorflow, Caffe, Mxnet, PyTorch, and others.

Instructors

  Tianqi Chen
  Haichen Shen
  Arvind Krishnamurthy


Teaching Assistant

  Qiao Zhang


Logistics

  Time: Tue/Th 11:30am - 12:50pm
  Place: Mueller Hall 153
    
      We will be using CSE 305 for some of the guest lectures, checkout the schedule for details
    
  
  Slack: Join https://uw-cse.slack.com dlsys channel for course discussions and announcements


Prerequisites

  Proficiency in Python, familar in C/C++
    
      We will mainly be using python for case study the existing systems,
and C/C++ for some of the background hacking.
    
  
  Basic knowledge of machine learning (e.g CSE 546)
  Prior knowledge in system (operating system/database) is useful but not required.


Homeworks and Grading

We will have assignments and a final project.


  Course project: 60%
  Homeworks: 30%
  Discussion participation: 10%


Acknowledgement
This course is created with help from DLsys seminar group at University of Washington.

  