Weight pruning of CNN
The short answer is both. 

Filter pruning

One approach is to remove filters that are "less contributing" to the overall learning every epoch or so. Defining "less contributing", that's the interesting bit about this research. For example it could be based on the running mean average of each filter, and remove filters of CNNs that have smaller mean. Some other research out there,


Using L1 norm to prune filters paper
This article list several other approaches (https://jacobgil.github.io/deeplearning/pruning-deep-learning)


Dense layer pruning

Obviously the parameter bottleneck of pretty much all CNNs happens at the first dense layer connected to the output of the last convolution layer. Therefore, doing proper pruning on this layer will be quite effective.

I couldn't find any good papers that introduces parameter compression for dense layers. But I'll update my solution if I come across any.

Knowledge distillation

Another really cool approach is known as knowledge distillation. Knowledge distillation is when you train a smaller network to mimic a larger network. One popular paper is found here.