Convolution Kernels for Image Filtering
Be careful, a median filter cannot be expressed as a convolution, and thus is not considered a kernel in this respect. This is because the median filter is based on order statistics of an image patch, and the resulting pixel at the output of a median filter is not a linear combination of other pixels within a patch. 

Otherwise, you are right, kernels are generally thought of as just other 'small images', that one would convolve with the image to be analyzed. Those kernels can be chosen for a variety of reasons. Gaussian kernels are used for smoothing, laplacians are used for edge detections, etc. 

There are as many kernels as you can imagine. However usually we start with a particular application in mind, "I need to blur the image", or "I need to detect edges in the x-direction", and then find kernels that magnify those features we are after.