Pruning Deep Convolutional Neural Network Using Conditional Mutual Information
CNN;pruning;optimization;mutual information;Renyi entropy
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
Yes
No ethics review needed.
* Could you please comment how your method compares to [1] 
* Further, I would like to see the results from the appendix 


[1] Ganesh, Madan Ravi, Jason J. Corso, and Salimeh Yasaei Sekeh. "Mint: Deep network compression via mutual information-based neuron trimming." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
Although other methods have been proposed to prune DNNs based on the mutual information (e.g. [1],[2]), it might be that the proposed selection criterion has some novelty.

[1] Ganesh, Madan Ravi, Jason J. Corso, and Salimeh Yasaei Sekeh. "Mint: Deep network compression via mutual information-based neuron trimming." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
[2] Hussien, Mostafa, et al. "Small Contributions, Small Networks: Efficient Neural Network Pruning Based on Relative Importance." arXiv preprint arXiv:2410.16151 (2024).
The authors propose a structured pruning method to reduce the computational complexity of CNNs. It uses conditional mutual information (CMI) to select feature maps that can be pruned from the network and applies an iterative re-training to compensate for the pruning errors.
There are several weaknesses:
* The paper only promises results on CIFAR10. However, this is simply not enough for a compression paper these days. 
* The paper seems to be incomplete. Results are promised in the appendix, but the appendix is missing.
* The only results provided in the paper (Table 1 and Table 2) show variants of their own pruning method, but do not compare to other literature.
* all plots miss axis labels
* References are missing (e.g. reference of Renyi entropy)
Yes
No ethics review needed.
Please see weakness.
1. The method part has a lot of information. 
2. The metric of mutual information is rather new.
This paper proposes a novel approach to structured pruning of Convolutional Neural Networks (CNNs) using Conditional Mutual Information (CMI). The authors present methods for ranking feature importance using CMI, determining pruning cutoff points, and implementing pruning strategies in both forward and bi-directional manners. The work demonstrates large model size reduction with small accuracy loss on VGG16 with CIFAR-10.
1. The largest weakness is the experiments. I'm supervised to see that only the old network VGG16 is tested on a small dataset CIFAR-10. Normally, for model pruning, ResNet-50 on ImageNet is necessary. Recently, Vision Transformer is also critical to validate the algorithm. Missing these important experiments makes this paper far below the standard of ICLR.
2. Missing important references. Many recent papers in 2024, such as [a], are not included. The author should do a complete survey.
3. Performance is not good. In [a], there is no accuracy drop when pruning 60+% FLOPs. 


[a] Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Yes
No ethics review needed.
Please see the weaknesses.
1. Code is released.
2. The paper is generally well-written, and the approach is very comprehensive.
3. The experimental results show the effectiveness.
This paper proposes a structured pruning method to achieve the compact DNNs. The proposed method leverages the conditional mutual information to demonstrate the importance of filters in each layer. Then, either the Scree test or X-Mean clustering is used to determine the pruning cutoff point for removing unimportant filters. Moreover, two pruning modes (forward pruning and bi-directional pruning) are investigated in this paper.
1. This paper only conducts experiments on VGG-16 on CIFAR-10, hence the generalization capability of the proposed method seems to be limited. The authors should report the performance on larger datasets (such as ImageNet) and the models with different architectures (such as ResNet).
2. The paper lacks the comparison with other state-of-the-art pruning methods to highlight its effectiveness.
3. The algorithm seems to be quite complex, as each layer requires numerous iterations to calculate CMI and the cutoff point. Additionally, it requires a large memory footprint to store the feature maps from previous layers for calculating cross-layer CMI. Does this imply that the algorithm may be difficult to deploy in compressing deeper networks or those with larger input size?
4. In Algorithm 4, when $k=1$, what does $F ^ s _ {0}$ represent?
5. In the experiment, a batch of 256 training samples is used to measure the filters. Does the number of samples affect the stability of the measurement? Can different samples produce the similar measurements? When more samples are used to calculate the measurement, can a more stable measurement be obtained?
6. Why is the pruning efficiency determined by the percentage of pruned filters over all filters (Line 468 on Page 9)? 

\
Minor issues:
1. $L _ i$ and $L_k$ should be unified in the first paragraph of Section 3.1.
2. $F ^ s _ 1$ in Line 1 of Algorithm 1 is repeated.
3. There is a symbol error in Line 241 on Page 5.
4. 'k' should be 'K' in Line 6 of Algorithm 2.
5. The references are repeated from Line 648 to Line 655 on Page 13.
@misc{
vu-van2024pruning,
title={Pruning Deep Convolutional Neural Network Using Conditional Mutual Information},
author={Tien Vu-Van and Dat Du Thanh and Nguyen Ho and Mai Vu},
year={2024},
url={https://openreview.net/forum?id=N0vzm0vwyR}
}
Convolutional Neural Networks (CNNs) achieve high performance in image classification tasks but are challenging to deploy on resource-limited hardware due to their large model sizes. To address this issue, we leverage Mutual Information, a metric that provides valuable insights into how deep learning models retain and process information by measuring the shared information between input features or output labels and network layers. In this study, we propose a structured filter-pruning approach for CNNs that identifies and selectively retains the most informative features in each layer. Our approach successively evaluates each layer by ranking the importance of its feature maps based on Conditional Mutual Information (CMI) values, computed using a matrix-based Rényi α-order entropy numerical method. We propose several formulations of CMI to capture correlation among features across different layers. We then develop various strategies to determine the cutoff point for CMI values to prune unimportant features. This approach allows parallel pruning in both forward and backward directions and significantly reduces model size while preserving accuracy. Tested on the VGG16 architecture with the CIFAR-10 dataset, the proposed method reduces the number of filters by more than a third, with only a 0.32% drop in test accuracy.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Tien_Vu-Van1 ~Dat_Du_Thanh1 ~Nguyen_Ho1 ~Mai_Vu1
Tien Vu-Van Dat Du Thanh Nguyen Ho Mai Vu
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
CNN pruning optimization mutual information Renyi entropy
I certify that there is no acknowledgement section in this submission for double blind review.
vuvan|pruning_deep_convolutional_neural_network_using_conditional_mutual_information
/pdf/e545e69b50da4cb37d9c255463660f35e1fb0896.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/d5428faad4951e6e668067c8d0c3bbff2ddb4e52.pdf
Pruning Deep Convolutional Neural Network Using Conditional Mutual Information
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission