JPEG Inspired Deep Learning
Deep Learning;JPEG Compression;Quantization;Non-linearity
Yes
No ethics review needed.
please see the weakness
1. Overall, the paper is written clearly, and can be easily understood.
2. A variety of experiments are conducted to verify the effectiveness.
3. The used technique seems sound, although I don’t check it in detail.
In this paper, the authors study the impact of JPEG compression to the performance of deep learning, and propose a JPEG-inspired deep learning framework. For that, they present a differentiable soft quantizer to train the JPEG layer. Experiments show that the proposed method increases the accuracy by almost 21%.
1. I don’t think the problem studied in this paper is very important in the community. Usually, the images fed into deep learning have been precessed by the fixed JPGE compressor, and we don’t have the chance to modify the process like that in this pager, so the actual application value is limited. In addition, I see the paper is an improvement for the method in Yang 2021 (Entropy), which is not followed by many researchers.  
2. Now lots of papers have shown the vision transformer and large model are effective in many computer vision tasks, I don’t know whether the proposed method can adapt to these new networks. The authors should discuss that. 
3. Although a variety of experiments are conducted, the compared method is only one baseline, which can not show the effectiveness.
Yes
No ethics review needed.
1. How to conduct datasets for experiments? For ImageNet-1k, the images are already compressed by JPEG. In traditional deep learning framework, the images are loaded and decoded by JPEG. How to insert the encoding operation in this framework?
1. To make JPEG trainable, a differentiable soft quantizer is proposed. It works well with JPEG. Overall, this paper makes JPEG trainable which is significant contribution. Because many frameworks equipped with JPEG can be trained by the differentiable soft quantizer.
2. A novel DL framework that prepends any underlying DNN architecture with a trainable JPEG compression layer is proposed. Experiments show it can improve the accuracy significantly with only 128 parameters.
3. This paper enjoys a good writing.
This paper proposes jointly optimizing both JPEG quantization operation and a DNN to achieve greater effectiveness. A trainable JPEG compression layer with a novel differentiable soft quantizer are proposed. Extensive experiments validate the effectiveness of the proposed method.
1. It is better to make a comparison for the latency. The speed of the model is also important to report.
2. Only image classification is considered. The proposed method is better to be validated on more tasks, such as object detection and segementation. 
3. Hyperparameters are tuned differently on different datasets.
Yes
No ethics review needed.
- Is the target of the adversarial attack in Figure 3 the whole model including the JPEG layer?
- There appears to be more room for exploration regarding preprocessing modules for performance enhancement. For example, what if deep learning-based image compression models were used? What about a simple autoencoder?
- The approach of leveraging image compression to improve pure performance of a model is interesting.
- The paper is well-written and it is easy to follow.
- A variety of network architectures and datasets are used in the experiments.
This work proposes a new training framework for deep learning models, JPEG-DL, utilizing an image compression module to improve the model performance. To this end, a learnable JPEG-based image compression layer is introduced with a differentiable soft quantization method and is trained jointly with a main model. At test time, a compressed image from the compression layer is fed into the model. The experimental results show consistent performance improvements of existing classification models on various benchmarks with higher robustness on adversarial attacks.
Major concerns:
- A comparison to training with JPEG-based data augmentation is required to validate that the proposed method provides benefits beyond simple data augmentation using JPEG. 
- There is no baseline for the differentiable quantizer. For example, comparisons could be made with methods such as the straight-through estimator (i.e., using the identity function as a gradient function) or additive uniform noise [1].
- The baseline for the image preprocessing method for training is insufficient; only comparison results with the vanilla models are presented. For stronger persuasiveness, comparison results with other learnable or non-learnable preprocessing modules are needed.
- In L199, it is mentioned that the differentiable soft quantizer is adapted from Yang & Hamidi (2024), but it seems unclear what that exact referenced paper is. Is the patent (https://patents.google.com/patent/US11461646B2/en) the correct source?
- The analysis on the significant difference between performance improvements across different datasets is needed. For instance, in Table 2, what accounts for the notable performance improvement in fine-grained tasks (especially the Flowers dataset)? Additionally, why is there few performance gain in the ImageNet results in Table 3?

Minor concerns:
- The empirical study is limited to classification tasks. 
- In Table 4, the bits per pixel (bpp) is excessively high, making it incomparable to typical lossy compression methods. While empirically showing the possibility of compression, it does not seem particularly convincing.
- The proposed method requires additional computation for encoding and decoding of an image, but the time complexity is not investigated.
- A typo in L292 "we fixe".

[1] Ballé et al., End-to-end Optimized Image Compression, ICLR 2017.
Yes
No ethics review needed.
* Could we see even better results if the JPEG layer was included periodically? 
    * What if *all* nonlinearity was replaced with the JPEG layer?
* Please clarify how $\alpha$ was used in experiments
* What about chroma subsampling?
* Why do the authors think this layer helps?
Overall this is a very interesting paper with an unintuitive result. As the authors point out (ln 30) the conventional wisdom is that JPEG compression removes information from an image and should only hurt neural network accuracy. However as this paper, and some prior works, show that is not necessarily the case. This paper builds significantly on prior works by showing not only that JPEG compression can be mitigated, but that it can actually be a large component of a neural networks success and proposing a method for achieving this. The differential soft quantizer is something which may be useful in many different applications, potentially being a better option that the addition of noise or a straight-through gradient as it more accurately models the information loss. Finally, the results show a clear improvement when incorporating the method.
The paper introduces a trainable JPEG inspired layer to neural networks. The new layer performs the linear JPEG steps of RGB to YCbCr and DCT with the non-differentiable quantization step replaced with a learnable quantization proxy finishing with the linear IDCT and YCbCr to RGB.  The learnable quantization proxy uses a soft-max with a tunable parameter which controls how close the proxy matches true quantization. The quantization step size is a learnable parameter in this layer. The paper shows how including this layer as the first layer of neural network architectures can improve their accuracy on different tasks.
While the appendix was fairly comprehensive with additional results there are a few additional things that I would have liked to see. The first is that the paper only tests the JPEG layer as the first layer of the architecture, there could have been more experiments in layer placement that would have been really interesting to see. It also wasn't immediately clear to me how $\alpha$ was being set in experiments, I understand that there is a derivation of $\frac{\partial}{\partial\alpha}$ but is that parameter actually trained and if so how was the gradient magnitude controlled? There is some discussion of this from ln 689 but it was a little unclear if $\alpha$ was fixed or not. One thing missing from the JPEG step was chroma subsampling: another non-linearity. Was this considered? It would be fascinating to see if neural networks respond to missing color information similarly to humans.

Lastly, and maybe most importantly, there was little discussion of *why this couterintuitive results holds*. While many view JPEG as something incidental the core idea of JPEG to isolate important information based on frequency bands. My take on the results presented here is that the learnable layer is essentially filtering out information which is irrelevant for the networks task, but I would love to hear the authors take on it. Perhaps such analysis could lead to a more direct approach? (For example: a layer which only filters frequencies or which alters the color channels, etc.)
We propose JPEG-DL, a new DL framework where a trainable JPEG compression layer is jointly optimized with DNN to improve the model prediction performance.
@inproceedings{
anonymous2024jpeg,
title={{JPEG} Inspired Deep Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=te2IdORabL},
note={under review}
}
Although it is traditionally believed that lossy image compression, such as JPEG compression, has a negative impact on the performance of deep neural networks (DNNs), it is shown by recent works that well-crafted JPEG compression can actually improve the performance of deep learning (DL). Inspired by this, we propose JPEG-DL, a novel DL framework that prepends any underlying DNN architecture with a trainable JPEG compression layer. To make the quantization operation in JPEG compression trainable, a new differentiable soft quantizer is employed at the JPEG layer, and then the quantization operation and underlying DNN are jointly trained. Extensive experiments show that in comparison with the standard DL,  JPEG-DL delivers significant accuracy improvements across various datasets and model architectures while enhancing robustness against adversarial attacks. Particularly, on some fine-grained image classification datasets, JPEG-DL can increase prediction accuracy by as much as 20.9%. Our code is available on https://github.com/JpegInspiredDl/JPEG-Inspired-DL.git.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Deep Learning JPEG Compression Quantization Non-linearity
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ed27a2428e7bdec196dd1c8b776ce0df7ce63a87.pdf
unsupervised, self-supervised, semi-supervised, and supervised representation learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
JPEG Inspired Deep Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission