Why is the kernel of a Convolutional layer a 4D-tensor and not a 3D one?
The first formula you quote is for an image with one input channel and one output channel, it just focuses on height and width. In this case, if we consider a 5x5 convolution, the Kernel will just have size 5x5, $m$ and $n$ and going from -2 to +2.
Now if our input has 3 channels (RGB, but could be feature maps). we need to use each channel as an input, and the weights will be different for each input map. So K becomes 3-D : 3x5x5. This new dimension corresponds to the $l$ in your formula.
If we want to have 10 outputs, we need 10 different ways of convoluting the input feature maps, so our Kernel will have one more dimension, leading to a 10x3x5x5 size for the Kernel. This last dimension corresponds to the $i$ of your formula.
To recap, the 4 dimensions of the Kernel $i, l, m, n$ stand for :

$i$ : Output dimension
$l$ : Input dimension
$m, n$ : height and width

About this sentence &quot;giving the connection strength between a unit in channel $i$ of the output and a unit in channel $j$ of the input&quot;.
I feel it is wrong as I would rather interpret it as : &quot;giving the connection strength between a unit in channel $i$ of the output and a unit in channel $l$ of the input&quot;