Preprocessing a video in android for pytorch
What is the best way to preprocess video data in Android Kotlin, in preparation to feed into a PyTorch Android model? Specifically, I have a ready-made model in PyTorch, and I've converted it to be ready for PyTorch Mobile.
During training, the model takes in raw footage from phones and is preprocessed to (1) be greyscale, and (2) compressed to a specific smaller resolution that I have specified, (3) converted to a Tensor to be fed into a neural net (or potentially send the compressed video to a remote server). I use OpenCV for this, but I'm wondering what the easiest way to do this would be in Android Kotlin?
Python code for reference:

def save_video(filename):

    frames = []

    cap = cv2.VideoCapture(filename)
    frameCount = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    frameWidth = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    frameHeight = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

    buf_c = np.empty((frameCount, frameHeight, frameWidth, 3), np.dtype('uint8'))
    buf = np.empty((frameCount, frameHeight, frameWidth), np.dtype('uint8'))

    fc = 0
    ret = True

    # 9:16 ratio
    width = 121
    height = 216
    dim = (width, height)

    # Loop until the end of the video
    while fc &lt; frameCount and ret:
        ret, buf_c[fc] = cap.read()

        # convert to greyscale
        buf[fc] = cv2.cvtColor(buf_c[fc], cv2.COLOR_BGR2GRAY)

        # reduce resolution
        resized = cv2.resize(buf[fc], dim, interpolation = cv2.INTER_AREA)

        frames.append(resized)
        fc += 1

    # release the video capture object
    cap.release()

    # Closes all the windows currently opened.
    cv2.destroyAllWindows()

    return frames


You said that your model was converted to be ready for PyTorch Mobile so I will assume that you scripted your model with TorchScript.
With TorchScript, you can write preprocessing logic using Torch operation and keep it inside the scripted model like this:
import torch
import torch.nn.functional as F

@torch.jit.script_method
def preprocess(self,
               image: torch.Tensor, # This should have format HxWx3
               height: int,
               width: int) -&gt; torch.Tensor:
    img = image.to(self.device)

    # (1) Convert to Grayscale
    img = ((img[:, :, 0] + img[:, :, 1] + img[:, :, 2]) / 3).unsqueeze(-1)

    # (2) Resize to specified resolution
    # Mimic torchvision.transforms.ToTensor to use interpolate
    img = img.float()
    img = img.permute(2, 0, 1).unsqueeze(0)
    img = F.interpolate(img, size=(
        height, width), mode=&quot;bicubic&quot;, align_corners=False)
    img = img.squeeze(0).permute(1, 2, 0)
    # Then turn it back to normal image tensor

    # (3) Other normalization like mean substraction and convert to BxCxHxW format
    img -= self.mean_tensor  # mean substraction
    img = img.permute(2, 0, 1).unsqueeze(0)
    return img

So all the preprocess will be done by libtorch, not opencv.
