Face Recognition with Video File I have built a face recognition model based on Pyimagesearch Tutorial

It works great on images but the issue I am facing is with videos.

I am recognizing the faces frame by frame(5fps). It gives me people present in the particular video frame.

But the same person is identifed with different names in different frames. 

How can I restrict it to identify same name with throughout the video? 

Sometimes it also recognized two different people with same name.
Eg. Sam and Dean are present in the frame but it identifies both as sam.
The pyimagesearch face recognition implementation used knn classifier to recognise the faces. I have worked with both svm and knn classifiers, from my experience you can do a couple of thinks to improve the face recognition performance. 
1. use svm with higher C value (1 to 10) over knn classifier.
2. Maintain relatively equal number of images in all classes