Deep Learning For Dummies

• Models
  • Deep Neural Network
    • Model 1
  • Convolutional Neural Network
    • Model 1
  • VGG16 Architecture
    • Model 1
    • Model 2
    • Model 3
  • UNET Architecture
    • Model 1
    • Model 2
    • Model 3
• Courses
• Resources
  • TensorFlow
  • Keras
  • PyTorch
  • PyTorch Lightning

Models

Deep Neural Networks

Model 1 (DNN)

• Deep Neural Network created using TensorFlow/Keras and the MNIST dataset. 60,000 images each 28 pixels x 28 pixels with 10 possible classifications.
  • Layers
    • Input Layer: 784 Nodes
    • Hidden Layer 1: 512 Nodes
    • Hidden Layer 2: 512 Nodes
    • Output Layer: 10 Nodes
  • Details
    • Optimizer: adam
    • Loss Function: categorical_crossentropy
  • Accuracy
    • 98.12 %

Convolutional Neural Networks

• Convolutional Neural Networks (CNN) — Architecture Explained
• VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION

Model 1 (CNN)

• Convolutional Neural Network created using TensorFlow/Keras and the MNIST dataset. The weights are preloaded from a file due to the time taken for each epoch.
  • Layers
    • 28x28x1
      • Convolving: 5x5x32
    • 24x24x32
      • Pooling 2x2
    • 12x12x32
      • Convolving: 5x5x64
    • 8x8x64
      • Pooling: 2x2
    • 4x4x64 (Input)
    • Hidden Layer: 1024 Nodes
    • Output Layer: 10 Nodes
  • Details
    • Optimizer: adam
    • Loss Function: categorical_crossentropy
  • Accuracy
    • 99.30 %

VGG-16 Architecture

Model 1 (VGG16)

• VGG-16 Pretrained Predictor Model with ImageNet created using Keras with an imported version of VGG-16 and imported weights. It trains the model with the ImageNet dataset and enables you to pass an image into the model. It tries to predict what the image is and classify it based on the categories offered by ImageNet.

Model 2 (VGG16)

• VGG16 Adaptations uses the CIFAR10 dataset. There are multiple models that are meant to mimic VGG16, but are slightly adapted due to the image sizes being 32x32 versus 244x244. I experiement with Convolutional Layers, Batch Normalization, Dropout, and Batch Size while trying to beat the imported VGG16 model.

Model 3 (VGG16)

• VGG16 from Scratch uses PyTorch and the CIFAR10 Dataset. This is the best example of a Convolutional Neural Network implemented using PyTorch. Great reference for future models.

UNET Architecture

• UNET Architectures are used for Image Segmentation

Model 1 (UNET)

• Oxford Pets is a dataset with numerous types of pets and usesd a UNET architecture in TensorFlow to segment images.

Model 2 (UNET)

• Carvana Image Segmentation is a dataset with numerous types of cars and is famous example for image segmentation. It uses PyTorch and an UNET architecture (no PyTorch Lightning).

Model 3 (UNET)

• Oxford Pets Part 2 is using PyTorch Lightning! Make sure to add Jacard and Dice Number for better metrics.

Courses

• Neural Networks and Convolutional Neural Networks Essential Training

  • Introduction to Neural Networks (Fundamentals, XOR Challenege)
  • Components of Neural Networks (Activation Functions, Visualization)
  • Neural Networks Implementation in Keras
  • Convolutional Neural Networks + Keras (Convolutions, Zero Padding, Pooling)
  • Enhancements/Image Augmentation to Convolutional Neural Networks
  • ImageNet + VGG16
  • TensorFlow

• TensorFlow: Working with Images

  • Neural Networks and Images (TensorFlow Hub, Hyperparameters)
  • Transfer Learning
  • Monitoring the Training Process (ModelCheckpoint, EarlyStopping, Tensorboard
  • TensorFlow

• TensorFlow: Neural Networks and Working with Tables

  • Fashion MNIST and Neural Networks
  • Loss, Gradient Descent, and Optimizers
  • Tabular Data
  • TensorFlow

• PyTorch Essential Training: Deep Learning

  • Fashion MNIST and Neural Networks
  • Classes and Tensors
  • Loss, Autograd, and Optimizers
  • CPU/GPU
  • PyTorch

• Transfer Learning for Images Using PyTorch: Essential Training

  • Notes
  • CIFAR10, Pretrained VGG16, PyTorch basics

• DeepLizard Videos

  • Notes
  • PyTorch basics, classes, and layers

• PyTorch Image Segmentation Tutorial with U-NET: everything from scratch baby

  • PyTorch, UNET, Image Segmentation

• Getting Started with PyTorch Lightning for Deep Learning (INCOMPLETE)

  • PyTorch Lightning, Deep Learning

• Learn PyTorch for deep learning in a day. Literally. (INCOMPLETE)

  • Everything PyTorch - Fundamentals, Workflow, Neural Network Classification, Computer Vision, Custom Datasets

Resources

• Deep Learning State of the Art (2020)
• Complete MIT Deep Learning Course
• Deep Learning MIT Book
• OneNote

TensorFlow

• Documentation
• TensorFlow Playground
• TensorFlow Hub

Keras

• Documentation

PyTorch

• Documentation
• Learn PyTorch
• Training A Classifier

PyTorch Lightning

• Tutorials
• From PyTorch to PyTorch Lightning
• Examples of PyTorch -> PyTorch Lightning