Filter dimensionality in ConvNNs
The diagram below Example 2 in this CS231 lecture is a good visualization of how filters work: http://cs231n.github.io/convolutional-networks/. Since each layer of a ConvNet has a depth corresponding to the number of filters (in an input RGB image, for example, the depth of the input layer would be 3), each filter of size $k\times k$ applied to a layer of delth $d$ slides over a volume of $k\times k\times d$ (note that the filter stays at a constant depth).