Face Recognition (Scalability Issue) Background

I would like to build a face recognition model for registration and login for some kind of service. For example, using this approach (CNN + SVM). 

When a new user wants to register a service, the image of his/her face is recorded and the machine learning model is trained using these images. Then, when a person requests for the service, the model recognises if this person is a member or not.



Question

But when there is new user comes in for registration, the machine learning model has to be retrained and scans through all the previous images (or feature vectors). It seems that this approach has a scalability issue when the the number of users is large.

I have read through this post but my situation is not quite the same as the suggested answer, because my machine learning model aims to distinguish member vs non-member. Does anyone know how to tackle this scalability issue? Thanks.
The problem you describe may be tackled with online machine learning where you continuously update your model as new data arrive,  avoiding the computationally intensive part of retraining.

For deep neural networks, there is some work in this direction.

scikit-learn and Vowpal Wabbit also provide some online algorithms.