Resize High Resolution Camera Capture to Low Resolution Camera Capture?
If you are using a “single-shot” sensor you could simply just not debayer the raw channels.  That would cut your sampling without adding the extra overhead that is already baked into the image acquisition.  A benefit of this approach is that you could probably boost your frame rate at the same time.