Is this gap between foundation and siding supposed to be here? If not what should I fill it with?
You can buy rolls of 'bottle or bristle brush' type material in different diameters. Essentially plastic bristles with a twisted wire core, simply cut to length and pushed up into place. 

Originally designed to deter mice, but it'd work for crickets just as well.