How can I fixed the filter and Kernel Size of a CNN?
Since the images are just 4x4 in size, you can do the following :

You can resize the image to a much larger dimension like 28x28 and then use sharpen or histogram equalization to bring out the contrast. Then use a 3x3x16, 3x3x 32 kernel arrays in 2 convolutional layers. The rest is fully connected.

The images can be input as 4x4 images but use a shorter neural net. One convolutional layer with padding and 3x3x 32 shape would do.

Use something similar to google nets, inception module, in which output from multiple convolutions are combined.