deep learning for non-image non-NLP tasks?
Yes you can use deep learning techniques to process non-image data. However, other model classes are still very competitive with neural networks outside of signal-processing and related tasks.

To use deep learning approaches on non-signal/non-sequence data, typically you use a simple feed-forward multi-layer network. No need for convolutional layers or pooling layers. The best architecture other than that needs to be explored with cross-validation, and can be time-consuming to discover as deep NNs take a lot of computation to train.

In my experience attempting to use deep(-ish, typically ~ 5 layers) neural networks in Kaggle competitions:


Dropout is still highly effective for regularisation and improving accuracy
Input normalisation - usually to mean 0, standard deviaton 1, is important
Hidden layer activation functions can make a difference. Although ReLU reduces some problems with vanishing gradients, in my experience it is less robust with non-signal data and you will want some other form. If you have only a few layers, then sigmoid or tanh still work OK. Otherwise, look into leaky ReLU, PReLU, ELU and other ReLU variants that attempt to patch its problems with "dead" neurons.
Make use of optimisers designed for deep learning, such as Adam, Adagrad or RMSProp
Use a weight initialisation approach that works with deep learning, such as Glorot.
Consider using Batch Normalisation layers. Not something I have much experience with, but I have seen other people do well with this approach.


Despite all this, XGBoost can routinely and easily beat deep NNs with minimal tuning and training effort in comparison (depending of course on the problem and the data you have). If accuracy is everything to you though, it is possible - although not guaranteed - that an ensemble of deep NNs and other model such as XGBoost will perform better than either singly.