Do I Need Pretrained Weights For Keras VGG16? As context, I am relatively new to the world of machine learning and I am attempting a project with a goal of classifying plays in an NBA game. My inputs are a sequence of 40 frames from each play in an NBA game and my labels are 11 all-encompassing classifications for a given play.

The plan is to take each sequence of frames and pass each frame into a CNN to extract a set of features. Then each sequence of features from a given video would be passed onto an RNN.

I am currently using Keras for most of my implementation and I chose to use a VGG16 model for my CNN. Here is some of the relevant code below:

video = keras.Input(shape = (None, 255, 255, 3), name = 'video')
cnn = keras.applications.VGG16(include_top=False, weights = None, input_shape=
(255,255,3), pooling = 'avg', classes=11)
cnn.trainable = True


My question is - would it still be beneficial for me to initialize the weights of the VGG16 ConvNet to 'imagenet' if my goal is to classify video clips of NBA games? If so, why? If not, how can I train the VGG16 ConvNet to get my own set of weights and then how can I insert them into this function? I have had little luck finding any tutorials where someone included their own set of weights when using the VGG16 model.

I apologize if my questions seem naive but I would really appreciate any help in clearing this up.
It would be beneficial to use pretrained weights trained on ImageNet because ImageNet is a huge dataset, and is fairly general. So you're more likely to get good image features from it. You might have to play around with from which layer you want to extract the features from.

If you want to create your own embeddings, you need to use the same code and instead of loading the weights, train it instead on the images you have. I do not recommend this for the following reasons


You most likely have too small a dataset for the model to converge  
VGG16 does classification, what will be the labels for the individual frames is a design choice and being a novice, it might be frustrating for you to get it right
It takes a lot of time and resources, not an ideal situation


P.S, if you still feel that ImageNet does not sufficiently cover your use case, you can try the pretrained Inception V3 model trained on the Open Images dataset