Introduction to Deep Learning

This is a short introduction to deep learning intended for those with no or little deep learning experience. Using Numpy and Pytorch, we cover the fundamental bases of deep learning: neural networks, backpropagation, and gradient descent. We also show practical examples of feedforward and convolutional neural networks on the Fashion MNIST problem.

This class is a part of the Reinforcement Learning Virtual School.

Installation

All course materials can be followed on Binder or through the Google Colab links below.

For a local installation of the materials, please install Jupyter. The slide format of the Jupyter notebooks is provided by the RISE plugin. This class is based on torch and torchvision; please refer to the PyTorch website for installation instructions. We’ll also be using the packages sklearn, numpy, and matplotlib.

Class Materials

All class materials can be downloaded from the Github repository.

We’ll be conducting a live poll throughout the class at the following link: https://ahaslides.com/DEEPINTRO.

1. Artificial Neural Networks
   • Github | Colab
2. Training ANNs
   • Github | Colab
3. Going deeper with layers
   • Github | Colab
4. Convolutional Neural Networks
   • Github | Colab

Additional Materials

This class is largely based on the Supaero Data Science Deep Learning class.

The deep learning book is fully available online and contains many great examples. Notebook versions of those examples are available here.

The Neural Networks and Deep Learning online book covers the universal approximation theorem with a nice interactive visualization.

PyTorch has a well-documented 60 minute blitz intended for beginners with PyTorch.

Stanford CS229 Lectures are freely available online. Lectures 11 and 12 introduce ANNs, backpropagation, and gradient descent, with lecture 12 going further in depth on how to resolve common training problems in ANNs.