Gone With the Bits: Revealing Racial Bias in Low-Rate Neural Compression for Facial Images
Fairness;Bias;Neural Compression;Phenotype Classification
Yes
None
No ethics review needed.
Weakness: (1)Although the author presents a novel topic, it seems that the author did not fully explore the way to solve the problem. Using a more balanced dataset seems to be one solution, but after discussion by the authors, this approach does not completely eliminate racial bias. So, how to better solve this problem? The author needs to give further elaboration. In fact, this is the point I am most concerned about. (2)The authors used traditional metrics such as PSNR and SSIM in their experiments to reflect racial bias. However, these metrics differ significantly from human visual experience. I wonder if the authors explored more perceptual metrics, such as LPIPS or FID?
Strength: (1)The topic of this paper is novel, and the racial bias of image compression on face data sets is studied (2)The authors have conducted quite sufficient experiments around this argument to verify that this problem does exist
This paper introduces a framework for assessing bias in neural image compression models, analyzing seven popular models and finding prevalent racial bias, manifested as unequal degradation of facial features. The study indicates that while using a racially balanced dataset helps mitigate bias, it is not a complete solution. The study indicates that while using a racially balanced dataset helps mitigate bias, it is not a complete solution.
Weakness: (1)Although the author presents a novel topic, it seems that the author did not fully explore the way to solve the problem. Using a more balanced dataset seems to be one solution, but after discussion by the authors, this approach does not completely eliminate racial bias. So, how to better solve this problem? The author needs to give further elaboration. In fact, this is the point I am most concerned about. (2)The authors used traditional metrics such as PSNR and SSIM in their experiments to reflect racial bias. However, these metrics differ significantly from human visual experience. I wonder if the authors explored more perceptual metrics, such as LPIPS or FID?
Yes
No ethics review needed.
Please refer to above weakness part.
1.	The paper has a clear problem definition, constructed a reasonable evaluation framework.
2.	Existing experiments have proved the existence of the problem from multiple angles to a certain extent
This paper investigated the racial bias problems existing in learned image compression. The authors built a framework to systematically examine the extent to which racial bias occurs in compression. Based on the evaluating framework, they proposed a classification-accuracy-based loss function to better reveal the bias. The correlation between bias, model architecture and image realism has been measured. They also show that utilizing a racially balanced training set cannot fix the problem.
1.    The paper seemly shows few contributions to the compression community. Since the paper just proposes the racial bias problem existing in learned image compression but provides no solution from the compression perspective. Similar evaluation schemes seem to be applicable to any field. Have you considered proposing compression-specific bias mitigation techniques?
2.    It seems that the bias problem is mainly attributed to the dataset and optimizing method. But the authors only focus on the data-related reasons and do not explore the impact of model optimization methods on this issue. It seems unconvincing to simply attribute the difference in model bias to the difference in model architecture. Have you considered analyzing how different loss functions or training regimes impact bias in compression models?
3.    The authors did not provide bias analysis results for images decoded by traditional codecs like JPEG, HM and VTM. The optimization of traditional codecs is not affected by the distribution of the dataset and should not lead to bias. If this experiment can be provided, it will promote our understanding of this problem. Please estimate traditional codec results at equivalent bitrates using the same bias evaluation framework.
4.    The author used the accuracy of the classification model to evaluate the loss of image attributes at low bitrates. However, the classification model was learned on undistorted images, and whether it can accurately classify features on distorted images is unverified. Additional experiments should be conducted in this regard to enhance the persuasiveness of the bias-related conclusion. For example, you could compare classifier’s results with human evaluations on a subset of distorted images and report the accuracy.
Yes
No ethics review needed.
* How can we show better reliability of the classification metric?
* Could INR methods overcome potential bias in neural compression?
The work here is interesting and timely. As neural compression continues to approach a usable state, but has yet to be deployed in any meaningful way, it is extremely important to start considering any potential bias in trained models or inherent to the algorithms. This way, the community can develop mitigations and ensure that those mitigations are used in future products. Additionally, I think the metrics used by the method to quantify bias are sensible and the finding that traditional distortion metrics, such as PSNR, do not accurately capture bias is a good result. This also makes sense since many compression techniques use pixel-error metrics as their objective. 

Although there is some overlap with prior work, as discussed in the paper, I think there is significant value in extending the analysis to neural compression.
The paper presents an analysis of racial bias in different neural compression algorithms. To measure this, the method uses face/phenotype classifiers and measure how much the classification decision is affected by the neural compression algorithm. This is similar to a rate-distortion measure where distortion is classification error instead of pixel error. The paper shows a clear bias in neural compression algorithms when trained on imbalanced datasets which is only partially mitigated by using balanced data.
While the classification error metrics presented in the paper is a good start it may need some additional development to make it fully capture bias. As the authors point out: the metric is only as good as the classifier itself. If the classifier is not able to make reliable decisions then the metric could miss bias or overly assign bias. I think this topic deserves more attention. Additionally the sensitivity of classifiers to different frequency degradations (which are common for compression); this may also explain different classification results at low bitrates.

Finally, there is an entire class of compression algorithms based on Implicit Neural Representations (see SIREN [1] for one example) which train a neural compression model unique to each example. This kind of technique could help mitigate any bias but these were not tested in the paper.

1. https://arxiv.org/abs/2006.09661
Yes
No ethics review needed.
1. Bias is defined as the maximum difference in loss (Eq. 3 and Eq. 6). How to deal with the impact of extreme values on results and how well does this definition of bias reflect the overall dataset?
1. This paper investigate bias in neural compression models, bringing attention to an underexplored area of fairness in AI. The authors reveal clear biases, particularly in skin-type degradation. 

2. The raised issue is very noteworthy and worth investigating, as it holds certain value for the generalization and reliability of neural compression methods.
This paper investigates racial bias in neural compression models for facial image compression, particularly at low bitrates. The authors demonstrate that traditional distortion metrics are insufficient for capturing racial bias, which manifests in noticeable degradation of facial features, especially for darker-skinned individuals. They examine the relationship between bias, model architecture, and visual realism, and show that while balancing the training dataset can help reduce bias, it does not fully eliminate it.
1. One of the major concern is that: while the paper identifies the presence of bias, the essential reasons are not thoroughly explored and mitigation strategies suggested (like dataset balancing) are not shown to be completely effective.

2. The experiments demonstrate that balancing the training dataset can help but does not fully mitigate the bias.  If the dataset balance is not working well. Network architecture’s impact on bias could be critical and should be analyzed further.

3. Exploring the fundamental causes of bias is critical. For instance, what would the bias level and visualized results be like if the network is trained and tested on an Africa-only dataset?
We present a framework to evaluate racial bias in neural image compression models, showing that examining facial phenotype degradation reveals racial bias, and that a racially balanced training set reduces but doesn't fully eliminate this bias.
@inproceedings{
anonymous2024gone,
title={Gone With the Bits: Revealing Racial Bias in Low-Rate Neural Compression for Facial Images},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Dolm7rrrQd},
note={under review}
}
Neural compression methods are gaining popularity due to their impressive rate-distortion performance and their ability to compress data to extremely small bitrates, below 0.1 bits per pixel (bpp). As deep learning architectures, these models are prone to bias during the training process, potentially leading to unfair outcomes for individuals in different groups. In this paper, we present a general, structured, scalable framework for evaluating bias in neural image compression models. Using this framework, we investigate racial bias in neural compression algorithms by analyzing 7 popular models and their variants. Through this investigation we first demonstrate that traditional distortion metrics are ineffective in capturing bias in neural compression models. Next, we highlight that racial bias is present in all neural compression models and can be captured by examining facial phenotype degradation in image reconstructions. Additionally, we reveal a task-dependent correlation between bias and model architecture. We then examine the relationship between bias and realism in the image reconstructions and demonstrate a trade-off across models. Finally, we show that utilizing a racially balanced training set can reduce bias but is not a sufficient bias mitigation strategy.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Fairness Bias Neural Compression Phenotype Classification
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/f050f0e5aaa75164086734a274c629cc1ffe52c9.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Gone With the Bits: Revealing Racial Bias in Low-Rate Neural Compression for Facial Images
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission