Training a CNN with limited weight sharing
It's important to note that before doing this, they split the frames into individual feature maps.  Note Fig. 1:


The "45 rows" in the upper left of your diagram (Fig 4.) corresponds to 15 frames times 3 feature maps.

The convolution is a 1D convolution, so they are not actually convolving across multiple frames, only across frequencies.  The authors then sum across feature maps in the normal way, meaning summing across frames and features.

Another way of looking at it is in Eq. 9



The variable $O_i$ is a vector across frequency bands only.  The sum is over feature maps which includes frames.