Structure of Convolutional Neural Network to analyze a sequence of frames
I'm not sure if I have to use a TimeDistributed layer or not


You definitely don't have to use TimeDistributed. You have other choices, that may be equally valid, depending on your data:


Flatten your example data into 81920 features per example and use a simple Dense layer.
Use a Conv3D layer.
Use some form of RNN, such as LSTM.


From your data description, I would expect either a TimeDistributed or 3D CNN based approach to be a good first bet. Intuition suggests that the CNN would work better if there was minor change between frames (because it has capability to directly find subtle frame differences), the TimeDistributed approach would work better processing larger changes (because it will ignore frame differences until the fully-connected layer).


  Should I use a TimeDistributed layer?


Only you can answer this, by trying it and measuring performance of your classifier. However, it should function correctly, and intuition suggests it would be a good choice if your frames are in sequence but visually disjoint.