Keras: apply multiple filters to each feature map in CNN
You may try Keras DepthwiseConv2D layer

Depthwise Separable convolutions consist of performing just the first step in a depthwise spatial convolution (which acts on each input channel separately). The depth_multiplier argument controls how many output channels are generated per input channel in the depthwise step.

It will convolute each Channel separately. As shown in this depiction.

$\hspace{6cm}$Image credit - Blog by Chi-Feng Wang
With depth_multiplier argument, you can add more Filetrs i.e. more copy of the &quot;triplet&quot; shown in the depiction.

depth_multiplier:
The number of depthwise convolution output channels for each input channel. The total number of depthwise convolution output channels will be equal to filters_in * depth_multiplier.