Zero-shot Mixed Precision Quantization via Joint Optimization of Data Generation and Bit Allocation
Zero-shot Quantization;Post-training quantization;mixed-precision quantization
Yes
No ethics review needed.
Some of the suggestions for clarity mentioned above.
1.	Proposes a unique, joint optimization approach for data generation and bit allocation, filling a gap in zero-shot mixed-precision quantization research.
2.	Demonstrates superior or comparable performance to existing state-of-the-art methods under low-bit settings and varied model budgets, with results verified on multiple architectures (ResNet-18, ResNet-50, MobileNetV2).
3.	As a complex system, the paper offers a well-rounded suite of ablation studies that demonstrate the robustness of the method
The paper presents a novel framework for zero-shot mixed-precision quantization (ZMPQ) that combines data generation and bit allocation optimization, focusing on improving quantization outcomes for deep learning models without access to original data. The approach introduces Gram-Gradient matrix-based data optimization and a scalable quadratic optimization for bit-width allocation, outperforming state-of-the-art methods on ImageNet benchmarks.
I am not familiar with the work related to Mixed-precision quantization task. This paper is intuitive and reasonable on the whole. However there are some problems of clarity in the writing
1. Certain proofs, such as the explanation of Equation (6) and the construction and application of the Gram-Gradient matrix, are highly technical, demanding considerable background knowledge from the reader. The explanation provided for Equation (6), in particular, feels insufficient for those not deeply versed in the topic.
2. The paper would benefit from additional figures or qualitative results to aid in understanding complex concepts such as gradient matching, which currently lack sufficient illustration.
3. Table 1 and 2, could provide more detail on the W/A notation (e.g., clarifying 2/2 and */2) to enhance interpretability.
Yes
No ethics review needed.
1. The references are all published until 2023. It would be best to provide some references on Mixed-precision Quantization/Zero-shot Quantization in 2024.
2. In Section 3.1, the authors mentioned that “In the realm of zero-shot quantization, the
validation set does not exist, so we only assume it here for explanation”. But the evaluation of model performance is conducted on the ImageNet dataset. Please carefully explain the relation between these two sets. If the validation does not exist, will there be significant changes or simplifications in formulas 6 and 8?
3. Please further explain the derivation process from formula 5 to formula 6.
4. In table 1 & table 2, the bit widths of results obtained using methods from this paper are marked as (*). From quantization setting in section 4.1, weights of some layers are set to 8 bits for initialization. Please calculate the average bit width of these models and fill it in the table, which will be more conducive to comparing the results and demonstrating the superiority of the method.
5. It would be better if the paper could compare the data synthesized in this paper with the data synthesized by existing methods and give deeper explanations or analyses.
The inherent correlation between the quality of the generated calibration dataset and the bit allocation to the model's layers are considered. The paper is well-organized and clearly stated. I would suggest accepting it after the following concerns are addressed.
This paper for the first time proposes a mechanism that combines zero-shot quantization and mixed-precision quantization.On the basis, the paper proposes a joint optimization framework between bit-width allocation and synthetic data generation. However, during the entire optimization process, these two parts are carried out alternately and iteratively. There is still a lack of a deeply coupled mechanism for joint optimization.
1. The references are all published until 2023. It would be best to provide some references on Mixed-precision Quantization/Zero-shot Quantization in 2024.
2. In Section 3.1, the authors mentioned that “In the realm of zero-shot quantization, the
validation set does not exist, so we only assume it here for explanation”. But the evaluation of model performance is conducted on the ImageNet dataset. Please carefully explain the relation between these two sets. If the validation does not exist, will there be significant changes or simplifications in formulas 6 and 8?
3. Please further explain the derivation process from formula 5 to formula 6.
4. In table 1 & table 2, the bit widths of results obtained using methods from this paper are marked as (*). From quantization setting in section 4.1, weights of some layers are set to 8 bits for initialization. Please calculate the average bit width of these models and fill it in the table, which will be more conducive to comparing the results and demonstrating the superiority of the method.
5. It would be better if the paper could compare the data synthesized in this paper with the data synthesized by existing methods and give deeper explanations or analyses.
Yes
No ethics review needed.
1. The quantizer setup. The GDFQ/Qimera/AdaDFQ series compared in this paper all use perchannel quantizer and naive optimization by STE. However, these details are not elaborated for the proposed method.
2. The proposed method also works for conventional PTQ when the calibration data set is assumed not to fit the model statistics, which indicates that the method is not completely limited to ZSQ. Authors should provide more targeted explanations or provide experiments of the proposed method on PTQ for corroboration. Perhaps the proposed method can be used to build benchmarks on PTQ?
3. The traditional benchmark on CIFAR data set has not been adopted, and the authors are requested to add explanations.
4. line235: "When that happens, we call the set X^(T) is k-equivalent to X^(V) . We can optimize this objective, by matching the Gram-Gradient matrix of the two sets, according to Definition 1 and Theorem 3.1" ZSQ does not provide a validation set. I am trying to doubt the assumption in the article that when two samples are from synthetic datasets, their matching does not indicate the correctness of the feature?
5. The generator configuration is not well aligned with the comparison method because Genie's generator is a newer version and this part needs further ablation.
6. Lack of overhead analysis.
The authors propose a mixed precision quantization method for ZSQ from the perspective of data quality. I support this work on the good topic.
They introduce a jointly optimization framework for zero-shot quantization that alternatively optimize the calibration data and model bit-widths
My main concern is that network optimization from a gradient perspective may be passive given the poor quality of the available data, and the intuition is that bypassing the gradient information and using a gradient-free search method may be a better option.
A jointly optimization framework for post-training quantization that alternatively optimize the calibration data and model bit-widths
@inproceedings{
anonymous2024zeroshot,
title={Zero-shot Mixed Precision Quantization via Joint Optimization of Data Generation and Bit Allocation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=OCHSgafZ1Y},
note={under review}
}
Mixed-precision quantization (MPQ) aims to identify optimal bit-widths for layers to quantize a model.
On the other hand,
zero-shot quantization (ZSQ) aims to learn a quantized model from a pre-trained full-precision model in a data-free manner, which is commonly done by generating a synthetic calibration set used for quantizing the full-precision model. While it is intuitive that there exists inherent correlation between the quality of the generated calibration dataset
and the bit allocation to the model's layers, 
all existing frameworks treat them as separate problems. This paper proposes a novel method that jointly optimizes both the calibration set and the bit-width of each layer in the context of zero-shot quantization. Specifically, we first propose a novel data optimization approach that take into consideration the Gram-Gradient matrix constructed from the gradient vectors of calibration samples. We then propose a novel  scalable quadratic optimization-based approach to identify the model's bit-widths. These proposals will then be combined into a single framework to jointly optimize both the calibration data and the bit allocation to the model's layers.
Experimental results on the ImageNet dataset demonstrate the proposed method's superiority compared to current state-of-the-art techniques in ZSQ.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Zero-shot Quantization Post-training quantization mixed-precision quantization
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/542459dfd7e3e36616814722d32333918c6fe818.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Zero-shot Mixed Precision Quantization via Joint Optimization of Data Generation and Bit Allocation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission