Why a convolution neural network model which works well on new test images fails on video stream?
Assuming the neural network works well on images, it should work the same on frames of a video stream. In the end, a video stream is a sequence of images.
The problem is not that it doesn't work on video stream, it simply does not work on the type of images similar to the ones you have in the video stream.
It is hard to exactly find the problem, since the question does not have enough detail. However, some considerations are:

Obviously, there is a problem with the network's ability to generalize. Was the testing performed well? For example, is there a train-validation split of the data?
Does the training error and the validation error indicate any possible issues, such as overfitting?
Is the data used to train the image similar enough to the video frames?
Is the training dataset large enough? Data augmentation might help if there is not enough data.
I have implemented a convolutional neural network by transfer learning using VGG19 to classify 5 different traffic signs. It works well with new test images, but when I apply the model upon video streaming it doesn't classify them correctly.