Featurizing images of different dimensions I'm building a non linear svm for images to solve a classification problem with domain {0, 1} and I'm currently doing featurization. What I want to do is create 3 features for each pixel representing the rgb value. However, my input images have different dimensions meaning that they are of different widths and heights. So doing just this would result in images having a different number of featurizations.

For example: If i have one image of (10, 2) dimension {(width, height)} then it has 10 * 2 pixels, each with a rgb value. So there would be 10 * 2 * 3 features for this image. If i have a (11, 2) image, then it has 11 * 2 * 3 features.

I am ordering the pixels by first moving horizontally and then verticall. So if I have a (2, 2) dimension image then F1, F2, F3 would be the first three features of pixel (0, 0) wheras F10, F11, F12 would be the features of pixel (1, 1)

One way I thought about solving this is by just extending each image to the maxwidth and maxheight over all images, filling values with -1's for rgb pixels where the image has been extended. For example if i have and image of dimension (2, 2) and the max width and height is (3, 3), then the new image would be of dimension (3, 3) and the with new pixels (2, 2), (0, 2) (1, 2) (2, 0) (2, 1). All of these new pixels would have -1 for all rgb values. 

However, will this affect my classifiers ability to recognize patterns? I'm planning on using a svm with radial kernal to start. Is there a better way to fix this?
Option 1: You can crop the images to the smallest sizes in all dimensions. However, blindly cropping images will cause you to lose important information, if you don't have a region of interest. For example, if you are focusing on faces, it is fine to define a ROI and crop around the face. On the other hand, for example if you are doing pure color-wise comparison of images, you will not know if you are cropping all red pixels from an image, that actually contain valuable information.

Option 2: You can scale them to the largest sizes in all dimensions. However, blindly resizing images might cause you to lose important structural information. For example, if you are doing color-wise comparison, you can upsample the images and the mean of the pixels will not change much. On the other hand, for example  if you are looking for faces in an image, your faces will not be circular but elliptical. 

Option 3: You can extend the images to the largest sizes in all dimensions. This introduces additional information if you don't filter it in the following steps. For example, for color comparison, you can fill with value nan and later neglect features which has value nan. You can fill with nan, or -1, or mean, etc.

Option 4: You can define a window to process the images. Imagine that you have an n*n window (say 3*3), and you slide that window on the image and extract features from each window (F1 is from [1:3,1:3] of the image, F2 is from [2:4,1:3] of the image, ...). Thus, the features are independent of the image size. However in this case you need to have a clever way to combine those features, because you will have different number of features from each image.