Deep Learning: Feed Forward for Unbalanced Classes Using Tensor Flow
To Improve the performance of the network, I added a Batch Normalization layer, and other variables that I had to remove when using Random Forest.  One of these variables is a categorical variable with 51 levels.  I made sure to add an embedding layer that will represent the values by a vector of size 10.
As the original dataset had 15 numerical variables and 9 categorical variables, I have decided to also perform dimension reduction using auto-encoding.