What is a deep neural network?
A deep neural network (DNN) is nothing but a neural network which has multiple layers, where multiple can be subjective.

IMHO, any network which has 6 or 7 or more layers is considered deep. So, the above would form a very basic definition of a deep network.