OV7670 frames are 640x308 not 640x480
For reference if others find this: The COM7 register was set to Bayer RAW (bits[2:0]=001). This causes the camera to output 640 by 480 bytes which can fit on the AL422B. However the Bayer interpolation needs to be done externally to get full rgb color picture. Alternatively the gains on r,g,b channels can be fiddled to get something close to a grayscale image which is what I desired.