Classifying ultrasound videos with a small dataset
One option in similar problems is to use transfer learning on raw images. For similar problems (few hundred images for training), solution that worked for me is transfer learning + image pre-processing to generate augmented images (E.g.: Stretched / Rotated versions of images).

High level design is :


Use a model that has been trained on large number of images (millions). This model has already learnt image processing as a domain (such as edge detection, phase / color changes and so on) and might remove need to feature engineering (like HOGS)
Add few more layers to be trained for Ultrasound images 
Train these new layers with images (Existing layers are frozen since they are already trained) 




Examples of this approach are :

https://www.researchgate.net/publication/311094437_Transfer_Learning_with_Convolutional_Neural_Networks_for_Classification_of_Abdominal_Ultrasound_Images

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5359213/

https://www.sciencedirect.com/science/article/pii/S2095809918301887