When is something a Deep Neural Network (DNN) and not NN?
You are right. Mainly any network with more than two layers between the input and output is considered a deep neural network. Libraries like tensorflow provide efficient architecture for deep learning applications such as image recognition, or language modelling using Convolutional neural networks and Recurrent neural networks. Another thing to keep in mind, is the depth of the network also has to do with the number of units being used in the layer. Mainly, as your non-linear hypotheses get complex you will need deep neural networks.