Issues with and alternatives to Deep Learning approaches? Over the last 50 years, the rise/fall/rise in popularity of neural nets has acted as something of a 'barometer' for AI research.

It's clear from the questions on this site that people are interested in applying Deep Learning (DL) to a wide variety of difficult problems.

I therefore have two questions:


Practitioners - What do you find to be the main obstacles to
applying DL 'out of the box' to your problem? 
Researchers - What
techniques do you use (or have developed) that might help address
practical issues? Are they within DL or do they offer an
alternative approach?
I have very little experience with ML/DL to call myself either practitioner, but here is my answer on the 1st question:

At its core DL solves well the task of classification. Not every practical problem can be rephrased in terms of classification. Classification domain needs to be known upfront. Although the classification can be applied to any type of data, it's necessary to train the NN with samples of the specific domain where it'll be applied. If the domain is switched at some point, while keeping the same model (NN structure), it'll have to be retrained with new samples. Furthermore, even the best classifiers have "gaps" - Adversarial Examples can be easily constructed from a training sample, such that changes are imperceptible to human, but are misclassified by the trained model.