Are the shortcomings of neural networks diminishing?
Neural Networks have other short comings as well. 


It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are resource constrained in anyway, you probably should not look at Neural Networks first. Evaluation of a trained deep NN can be much more expensive than competing techniques too.
The effort involved in learning how to architect and train a NN is still much higher than competing methods, like an SVM. People who are just starting out in Data Science should probably use other techniques to learn about the nuances of fitting data before getting involved in neural networks. And although simple NNs with only one or two hyperparameters are often available in 
many data science libraries, they don't perform any better than other techniques so are just another ML black-box technique really.
While we have made a lot of progress in understanding how neural networks do their magic, they are still less accessible and dissectible than most competing methods. So while NNs might solve the problem, they might not give you as many insights as easily as other techniques do.


Looking forward to what other people have to say here.