Analyzing Videos using Deep Learning
Yes there is. Actually using NNs on sequences is a big part of Deep Learning and one of the most powerful NNs come from this field of research. They are called Recurrent Neural Networks or RNNs. I would recommend you to read this article or watch this Stanford lecture, to learn more about them. 

Especially a variant of RNNs, the so called LSTM that stands for Long Short Term Memory enables networks to memorize earlier inputs.

Based on your explanation I would suggest you have a many-to-one task where you input many frames and output one class. These types of tasks where you input images is most often solved using a combination of CNN and RNN. You could read this article to get more insights, to that technique.