TODO LIST

1. Hands on exercise training NN using gradient descent
    https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
    DONE 09/22/2020
    
2. NN from scratch 
    
    a. using numpy
    https://repl.it/talk/announcements/Build-a-Neural-Network-in-Python/5457
    
    b. using pytorch
    https://medium.com/dair-ai/a-simple-neural-network-from-scratch-with-pytorch-and-google-colab-c7f3830618e0
    
4. Pytorch tensor 
    https://pytorch.org/tutorials/beginner/examples_tensor/two_layer_net_tensor.html
    
5. Image classifier
    
    a. Train on CPU / single GPU
    https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html
    
    b. Train on multiple GPUs
    https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html

More reading
   * https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/
   * https://www.activestate.com/blog/top-10-python-machine-learning-packages/
   * [Regularization](https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/)

 
Prepare exercises for NN implementations
Prepare exercises for NN implementations
