Smoothen the Classification output I am working on an image classification tasks in which I have 4 classes (For example A, B, C, D). I used CNN model (transfer learning) to train the model and predict the video frames.

Ideally, there would not be any sudden transition from one class to another. Ideal Transition would be as given below: 
A A A A A ..... A A A A B B B B B B ..... B B B B B B B D D D D D ..... D D D C C C C C C C ............. C C C

However when predicting using the trained model, I could see some frames being mis-classified. For example I given an video input of Class A which has around 30 frames. In that 30 frames 5 would be predicted as Class C or D. 

How would I make the smooth transition from one class to another. There should be sufficient number of evidences so that I can make transition from one class to another. 

As of now I found moving average technique which does similar kind of smoothening. I would like to know is there any other method which is more related to probability.

Kindly let me know if you need more details.

Thank you,
KK
An option could be using the output of the i-th image as an input for classifying the (i+1)-th image.