reduction of sample from videos sample
Reduce the size e.g. using cv2.resize()
Compress the image (it is not lossless) e.g. cv2.imencode()
Lower the frame rate
Use lower precision - images are uint8 when loaded, but the deep learning frameworks use float32 by default. You could try float16 or mixed precision.


Using JPEG compression has been shown to be fairly good in terms of the reduction in memory and minimal loss of performance. Have a look a this research.

You could also drop the frame rate, so say 10 FPS. The actually values could be computed based on the expected velocity of the moving objects -> do you really require 24 FPS for the task?

Otherwise, the hardware you are using will determine which steps to take afterwards. Memory, number of operations, inference speed etc. will change how you optimise the process.



You mentioned "dataframe", so I will just point out that using Pandas Dataframes to hold raw image data, whilst looking easy, it generally very inefficient due to the number of data-points involved (pixels), and the fact that Pandas DataFrames are essentially annotated NumPy arrays - the annotations take a lot of space. Better to load into pure numpy arrays and use OpenCV for things suchs as making gray-scale (black and white) images from RGB, resizing them, normalising pixel values, and so on.