Convolutional layer: does the filter convolves also trough the nlayers_in or it take all the dimensions?
In the leading DeepLearning libraries, does the filter (aka kernel or weight) in the convolutional layer convolves also across the &quot;channel&quot; dimension or does it take all the channels at once?
To make an example, if the input dimension is (60,60,10) (where the last dimension is often referred as &quot;channels&quot;) and the desired output number of channels is 5, can the filter be (5,5,5,5) or should it be (5,5,10,5) instead ?

It should be (5, 5, 10, 5). Conv2d operation is just like Linear if you ignore the spatial dimensions.
From TensorFlow documentation [link]:

Given an input tensor of shape batch_shape + [in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, out_channels], this op performs the following:

Flattens the filter to a 2-D matrix with shape [filter_height * filter_width * in_channels, output_channels].
Extracts image patches from the input tensor to form a virtual tensor of shape [batch, out_height, out_width, filter_height * filter_width * in_channels].
For each patch, right-multiplies the filter matrix and the image patch vector.


