What are some resources with exercises related to neural networks? I am asking for a book (or any other online resource) where we can solve exercises related to neural networks, similar to the books or online resources dedicated to mathematics where we can solve mathematical exercises.
The book Grokking Deep Learning, by Andrew Trask (a PhD student at Oxford University and a research scientist at DeepMind), a wonderful, clean, and plain-English discussion of the basic mechanics that go on under the hood of neural networks - from data flow to updating of weights.  It is written without a slant on normally-wonky math, the concepts are presented and then advanced at a digestible pace for anyone. 

Here are a few more possibly useful resources.


Neural Networks: Playground Exercises 
Getting Started With Deep Learning: Convolutional Neural Networks
Deep learning focuses on practical aspects of deep learning
First lab assignment in Deep learning
Deeplearning stanford