Does a Convolutional Layer in a Neural Network learn the correlation between its input signals via its kernel?
In a convolutional neural network (CNN), a convolutional layer has several channels, each of which has one convolution kernel, often written down as a matrix. This convolution kernel is nothing more than a collection of weights used to compute a linear combination of elements of the input.
While both &quot;traditional&quot; dense layers and convolutional layers compute a linear combination of their inputs, convolutional layers have the added structure of preserving spatial information. Additionally, the output of a convolutional layer, called a feature map, can be understood as the result of sliding the kernel along the input.
For instance, take a simple kernel $K = \begin{bmatrix} 1 &amp; 2 &amp; 3 \\ 4 &amp; 5 &amp; 6 \\ 7 &amp; 8 &amp; 9 \end{bmatrix}$ and a small input &quot;image&quot; $I = \begin{bmatrix} 9 &amp; 8 &amp; 7 &amp; 6 \\ 5 &amp; 4 &amp; 3 &amp; 2 \\ 1 &amp; 2 &amp; 3 &amp; 4 \\ 5 &amp; 6 &amp; 7 &amp; 8 \end{bmatrix}$. We take an input with only one channel for convenience: if it's the first layer of the CNN, we can assume the values of $I$ represent a grayscale image. Let us look at how the convolution is performed if no padding is added to the image, in which case the output will be a 2 by 2 matrix.
Starting at the top left corner, we convolve $K = \begin{bmatrix} 1 &amp; 2 &amp; 3 \\ 4 &amp; 5 &amp; 6 \\ 7 &amp; 8 &amp; 9 \end{bmatrix}$ with $\begin{bmatrix} 9 &amp; 8 &amp; 7 \\ 5 &amp; 4 &amp; 3 \\ 1 &amp; 2 &amp; 3 \end{bmatrix}$. To do so, we multiply the two matrices element-wise, then sum up all the elements of the resulting matrix: $1 \cdot 9 + 2 \cdot 8 + 3 \cdot 7 + 4 \cdot 5 + 5 \cdot 4 + 6 \cdot 3 + 7 \cdot 1 + 8 \cdot 1 + 9 \cdot 3 = 146$.
Doing the same three more times yields the following feature map: $\begin{bmatrix} 146 &amp; 157 \\ 200 &amp; 233 \end{bmatrix}$.
Depending on the kernel of a layer, the feature map will look different, and the information extracted from the input has a different &quot;meaning&quot;. The idea behind a CNN is to make the coefficients of the kernel learnable parameters. Often, multiple &quot;meanings&quot; must be extracted from a single input, so a convolutional layer has multiple channels.
However, CNNs didn't invent convolution, they merely made the kernels learnable. Indeed, such methods have been used in &quot;traditional&quot; computer vision (CV) for a long time. In traditional CV, the kernels are hand-crafted to fill specific roles, extract specific types of information.
For edge detection or line detection, you might use a Prewitt filter which is the discrete equivalent of a derivative operator or a slightly more complicated Sobel filter.
In fact, it is not uncommon for a CNN to learn approximations of well-known hand-crafted convolution kernels from traditional CV on the first layer. Also, by visualizing feature maps on deeper layers, as the training of a CNN progresses, gives you good insight into what the specific convolution channels are learning to detect.