Image Classification task with unevenly sized images
The size of the input image is usually decided by computational resources you have. If input image has larger dimension you need to increase filter size, then model will have more parameters, hence you need more computational resources and longer training time.
Usually, I resize the images to min(heights of all images), min(widths of all images).
If there are larger samples of particular dimension in the dataset and you have enough computational resource, it might also makes sense to resize to that dimension.
Try with different settings, pick the one that best works for you.