Face Recognition (Scalability Issue)
The problem you describe may be tackled with online machine learning where you continuously update your model as new data arrive,  avoiding the computationally intensive part of retraining.

For deep neural networks, there is some work in this direction.

scikit-learn and Vowpal Wabbit also provide some online algorithms.