Filter dimensionality in ConvNNs This is a major confusion for me - I've always thought filters (those small sliding windows size 3x3 or 5x5) are strictly 2D, this among other is specified in Caffe API: 


  kernel_size / kernel_h / kernel_w. The filter dimensions, given by
  kernel_size for square filters or kernel_h and kernel_w for
  rectangular filters


Now, I've never been 100% clear on what these convolutions mean (taken from Matt Zeiler's article (Visualizing and Understanding Convolutional Networks) and how they were obtained:
 

After searching it turned out that filters can be 3D?? How does it work then? I found a bit of information on CS 231n course, but it's not explained in detail anywhere.
The diagram below Example 2 in this CS231 lecture is a good visualization of how filters work: http://cs231n.github.io/convolutional-networks/. Since each layer of a ConvNet has a depth corresponding to the number of filters (in an input RGB image, for example, the depth of the input layer would be 3), each filter of size $k\times k$ applied to a layer of delth $d$ slides over a volume of $k\times k\times d$ (note that the filter stays at a constant depth).