How can I use deep neural networks to recognize characters on vehicle license plate?
Before jumping into any machine learning task it is good to consider your dataset and the features you wish to use. License plates are unique thus feeding the entire plate into a machine learning algorithm would not yield very accurate results. First, you will want to make sure that the extracted license plate images are similar in size. Then you will want to separate the license plate such that you can separate the different digits and letters into their own respective images. Thus your dataset that will be used to train your network will consists of many single character images which contain either a letter or a digit. When you have finished training your model, future license plates will also have to be broken down in this way, classified and then the outputs be combined to get the license plate number.

What is a neural network?

A neural network (NN) is a series of nodes which contain a simple, continuous, differentiable function such as logistic regression. Alone these nodes are not very powerful, however, the beauty comes in when they are connected in networks. The tuning of such a network can learn highly complex functions. Typically, a NN is set up with three layers. The input layer, where you will feed in your features, the hidden layer and the output layer. Naturally, if you want a deep NN then you just need to add more hidden layers. But, there is a trap. The deeper your network is the harder it will be to train, thus you will need MORE data. 

The input-layer

This is where you will be feeding in your images. If you have a 16*16 pixel image then you will have 256 input nodes. This layer typically does not perform any function on the input, it is simply taking in the input. It will then feed the input to the hidden layer. You can use some dimensionality reduction techniques to try and eliminate some useless information from your dataset such as the white spaces in the corner of the image. The less input nodes you have, the more informative your samples become, thus with higher information entropy your network will be able to learn faster, or more rigidly with the same amount of data. 

The hidden layers

You are free here to choose how many layers you want and how many nodes you want in each layer. I suggest using grid search and cross-validation as a means to determine which configuration is ideal for your dataset. Too many layers and nodes and you will not end up with a fully trained networks once you run out of training data (too much bias). Too little nodes, your network will not be able to learn the subtleties which are embedded within the data (too much variance). You want to find yourself at the crossroad where these two are minimized. Choose wisely. 

The output layer

This layer can be configured in a few different ways. Typically people like to do one output node for each class. In your case there are 26 letters in the alphabet and 10 digits. This gives 36 possible output classes. So you will need 36 nodes in your output layer.

How do I train this?

The training of the NN is by tuning the weights that will be applied to the inputs of each of your nodes. Try to draw your dense network, for each line connecting nodes you will have a weight. I am sure you can see how this becomes quite huge. You will initialize your weights randomly to start. Then you will take each of your examples and pass them through your network. This is called a feed-forward pass through. This will provide you with an output. You will compare your output with your ground truth value using a cost function. This is often the root mean squared error (RMSE) but there are many other options. Then using the derivative you will back-propagate the error through your network to see which weights caused the error and to what extent. The derivative will then allow you to update those weights proportionately.

WOW do I need to program all that?

Nope not anymore!!! You can use Keras in python (my recommendation) or other similar frameworks which have made it a real tea sipping pleasure to program a variety of NN configurations. 



My recommendation

I would start with a simple NN and see what kind of accuracy I can achieve. From there I would try to use convolutional neural networks which have been shown to garner much better results on datasets comprised of images. These can also be implemented in Keras. CNN require more data than typical NN thus you might need to get access to more images.

It is always best to train your dataset only on images you have in your original data. However, when it comes to images, adding to your data by artificially shifting images by 1-5 pixels or rotating by 1-5 degrees can add variability which can supplement your original data. 

Make sure that the license plate pictures you are using to train your network are taken with a similar system as the images you will be using in the future. You will not get very good results if you train your network on images taken directly behind cars and then you expect to implement your solution in traffic cameras monitoring traffic from above.