Introduction to Deep Learning

Deep Learning is a rapidly expanding field with new applications found every day. In this workshop, we will cover the fundamentals of deep learning for the beginner.

We will introduce the math behind training deep learning models: the back-propagation algorithm. Building conceptual understanding of the fundamentals of deep learning will be the focus of the first part of the workshop. We will then cover some of the popular architectures used in deep learning, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), LSTMs, autoencoders and GANs.

There will be a hands-on computing tutorial using Jupyter notebooks to build a basic image classification model via transfer learning. By the end of the workshop, participants will have a firm understanding of the basic terminology and jargon of deep learning and will be prepared to dive into the plethora of online resources and literature available for each specific application area.

About the Instructor

Aashwin Mishra is a Project Scientist at the Machine Learning Initiative at the National Accelerator Laboratory (SLAC). His research focuses on uncertainty quantification, probabilistic modeling, interpretability/explainability, and optimization across physics applications.

Workshop Materials

Pre-workshop Checklist

The workshop assumes that you have requisite knowledge of:

a) Python programming

b) Numpy

c) basic concepts of Machine Learning (desirable)

For (a), you should have written codes in Python, and at some juncture, defined and used a class.

For (b), you should be comfortable in basic linear algebra operations in Numpy, understand broadcasting, etc.

(c) is desirable, but not required.

Schedule

Session 1 (Tuesday, August 9 1:00 PM - 4:00 PM PDT)

-Recording of session 1

-Review of basic ML concepts

-Overview of deep learning

-Coding a perceptron in numpy

-Understanding fully connected neural networks

-Coding fully connected neural networks in numpy

-Introduction to Torch

-Using the AD facilities in torch to train models

-Using the sub-modules in torch to define and train models

Session 2 (Wednesday, August 10 1:00 PM - 4:00 PM PDT)

-Recording of session 2

-Understanding Convolutional Neural Networks

-Defining and Training CNNs in torch

-Using GPUs for training

-Data Augmentation

Google Colab notebooks

-Linear Regression Exercise in Numpy:

-Solutions: Linear Regression Exercise in Numpy

-Shallow Neural Network in Numpy:

-Solutions: Shallow Neural Network in Numpy

-Introduction to Torch

-My First Torch Model:

-MNIST with an FCNN:

-CIFAR10 with CNNs:

-CIFAR10 with GPUs:

-Data Augmentation for Computer Vision: