Deep Learning for Video Classification
My suggestion is to use Convolutional and Recurrent layers in the same Neural Network.

You'd have to capture a given number of frames of a video (let's say one each 0.5 seconds), and feed arrays of screenshots into the model. Its structure would be:


Conv (and MaxPool) layers to process pixel data - they will extract and process relevant information from each screenshot.
LSTM layers - that will process their sequence, extracting meaning from their flow.
Dense layers at the end to perform classification, with softmax activation at the output layer.


That's how I would do. It's going to be computationally expensive, if you don't have a GPU it won't be easy.