What are the disadvantages of using deep neural networks compared to a linear model?
First of all, we should say that a single affine layer of a neural network without any non-linearities/activations is practically the same as a linear model. Here we are referring to deep neural networks that have multiple layers and activation functions (non-linearities as relu, elu, tanh, sigmoid etc.)

Second of all nonlinearities and multiple layers introduce a nonconvex and usually rather complex error space which means that we have many local minimums that the training of the deep neural network can converge to. This means that a lot of hyperparameters have to be tuned in order to get to a place in the error space where the error is small enough so that the model will be useful.

A lot of hyper parameters which could start from 10 and reach up to 40 or 50 are dealt with bayesian optimization using Gaussian processes to optimize them which still does not guarantee good performance.

Their training is very slow and adding the tuning of the hyperparameters into that makes it even slower where in comparison the linear model would be much faster to be trained. This introduces a serious cost-benefit tradeoff.

A trained linear model has weights which are interpretable and give useful information to the data scientist onto how various features play a role for the task at hand.