Discovering Clone Negatives via Adaptive Contrastive Learning for Image-Text Matching
Image-text matching;Contrastive learning;Vision and language;Multimodal learning
Yes
No ethics review needed.
The method figure (Figure 2) is unclear, and neither the figure caption nor the text adequately explains what the "reference clone negatives" refer to.
1. The paper introduces a novel approach to address the issue of clone negatives in image-text matching, which is a pertinent problem in the domain.
2. The proposed Adaptive Contrastive Learning (AdaCL) framework dynamically adjusts the compactness between positives, showing potential for enhanced performance in image-text matching tasks.
This paper addresses a challenge in image-text matching referred to as clone negatives, which are negative image-text pairs that are semantically consistent with the positive pairs, leading to ambiguous and suboptimal matching results. To tackle this issue, the authors propose Adaptive Contrastive Learning (AdaCL), which employs two margin parameters with a modulating anchor to dynamically enhance the compactness between positives while mitigating the influence of clone negatives. The modulating anchor is selected based on the distribution of negative samples without explicit training, facilitating progressive tuning and improved in-batch supervision. Experiments on two benchmark datasets underscore the effectiveness of AdaCL.
1. The experimental results are not convincing. The main contribution of the paper is the improvement of Contrastive Learning, with CLIP being the most representative of this field. To prove the effectiveness of the paper, a comparison within the OpenCLIP framework using larger datasets (e.g., LAION) should have been conducted, rather than experiments on the relatively small-scale COCO dataset.
2. The proposed method appears to be somewhat complex, which goes against the inherently simple and scalable nature of contrastive learning. Additionally, the paper does not provide an analysis of the introduced hyperparameters or the potential increase in overall complexity.
3. The authors do not clearly explain why clone negatives pose a problem within the paradigm of contrastive learning. Clone negatives refer to texts that are aligned with the image but are not as accurate as the ground-truth text. Hence, the logits of clone negatives should naturally be lower than those of the ground-truth, which aligns with the basic optimization framework of contrastive learning. The authors should provide more empirical evidence or theoretical analysis demonstrating why standard contrastive learning fails to handle clone negatives adequately.
Yes
No ethics review needed.
Please refer to weakness above.
1. The authors focus on a critical issue of clone negatives in the field of Contrastive Learning under the background of Vision-Language Models and Image-Text Matching Tasks.   
2. Experiments are well performed, which show effectiveness of AdaCL.
This paper focused on solving the task of Image-Text Mathing (ITM) especially the key challenge of clone negatives: the captions that have consistent semantics with the GT sentences, but with less coarse-grained information. To solve this, the Adaptive Contrasvite Learning (AdaCL) method is proposed, which introduces supervision of the clone negatives, and can dynamically penalize the clone negatives. This will enlarge the distance between the positives and the clone negatives.
1. More pre-trained models and more down-stream tasks could be test to further demonstrate the effectiveness of the proposed method.

2. Spelling error(s): 
p2 077 andand
p2 Fig 1(b) T2 -> T3
p1 034 action-level -> global level
p6 274 double curly braces “}” -> just one “}”, delete the last one

3. Sequential adverbs: 
p4 205 There are “Two conditions”. I can see the “First”, but where is the “Second”?
It should be before p4 214 ”(Second, )to achieve this”. 

4. Meaning of subscript: 
p4 208 What does the subscript “u” mean? What is its abbreviation? Is it “unique(==specific)”?
And what about p4 210 the “u-” in k∈M_u-? Is it “not unique”? I can’t sure. 

5. Inconsistencie(s): 
p4 Sec 3.2 one anchor + one pos + M neg ==> Why not all M neg? Because Sec 3.2 and Sec 3.3 are connected. 
p5 Sec 3.3 one anchor + one pos + (M-1) neg
p6 Algorithm 1 a mini-batch of N pos + M neg??? ==> Why not one pos + M neg?
  May be the authors want to say “a mini-batch of N, in which 1 pos and M neg”?
  So, N==1+M?

6. p5 254 what is the superscript of j in Eq. (8)? Is it the same as Eq. (7) or Eq. (3)?

7. p5 263 Why Eq. (9) is that? Can you give us a concrete deduction? And what about p(C=0|s)? Can you give a specific formulation?  

8. p10 509 How to correctly understand the heat map, since a sentence contain ALL WORDS, and every mentioned instance in the image SHOULD be highlited. So, what does these Attention Map mean indeed? What Information can I get from them? Or what should we expect from them?
Yes
No ethics review needed.
1. From what I understand, this method is matching images with texts. Then why is FasterRCNN, an object detection model, used in the experiments?
1. The motivation of this work is clear. The issue of clone negatives is well illustrated and intriguing, which is common and significant in matching images and texts.

2. The proposed method is extensively verified under both supervised and weakly supervised settings, exhibiting superior performance under several benchmarks.

3. The authors provide comprehensive ablation studies and analyses, with both quantitative and qualitative verification.
This paper recognises the clone negative issue in image-text matching, where negative image-text pairs are semantically consistent with the positives. To address this issue, the authors modulate the softmax function of original contrastive losses with anchors that are selected from each in-batch similarity score based on Gaussian discriminant analysis. The proposed method, called Adaptive Contrastive Learning, demonstrates superiority in both supervised and weakly-supervised image-text matching.
1. The method is not applied to or compared with the recent CLIP-style models, which I believe would strengthen the significance and soundness of this work.
Yes
No ethics review needed.
See weakness.
1. The author's idea of ​solving the false-negative problem is clear.
2. The experiments are rich and the proposed method is effective.
The authors of this paper propose a new method to improve cross-modal learning by adding a conditioning factor to the loss function to alleviate what the authors call "clone negatives". The proposed method is effective from the experimental results. However, the concept of clone negatives is more like the False-negative problem mentioned in some existing studies. For this reason, the concept of clone negatives still needs further discussion.
1. The authors proposed a new concept, namely clone negative examples. However, this is actually the false-negative problem, which is not a new topic. For example, there has been similar research in [1,2].
2. The authors should enrich the caption of Figure 2 to make the core idea clear and concise.
3. Line325: “Comparisons of Image-text matching” should be “Comparisons of image-text matching”
4. The 2024 baselines are missing.
5. Line 954,955: Missing citations.
6. There are usually three mainstream benchmarks for text-based person search tasks. I suggest the author add the results of RSTPReid. Also, does the authors‘ R@ refer to the Recall accuracy? However, in the text-based person search task, the Rank accuracy is usually used as the evaluation indicator.

Reference:

[1] Li H, Bin Y, Liao J, et al. Your negative may not be true negative: Boosting image-text matching with false negative elimination[C]//Proceedings of the 31st ACM International Conference on Multimedia. 2023: 924-934.

[2] Li Z, Guo C, Feng Z, et al. Integrating language guidance into image-text matching for correcting false negatives[J]. IEEE Transactions on Multimedia, 2023, 26: 103-116.
We propose Adaptive Contrastive Learning (AdaCL) for image-text matching, which introduces two margin parameters with a modulating anchor to dynamically strengthen the compactness between positives and mitigating the impact of clone negatives.
@inproceedings{
anonymous2024discovering,
title={Discovering Clone Negatives via Adaptive Contrastive Learning for Image-Text Matching},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=My9MBsO41H},
note={under review}
}
In this paper, we identify a common yet challenging issue in image-text matching, i.e., clone negatives: negative image-text pairs that are semantic-consistent with the positives, which leads to ambiguous and suboptimal matching results. To address this, we propose Adaptive Contrastive Learning (AdaCL), which introduces two margin parameters with a modulating anchor to dynamically strengthen the compactness between positives and mitigating the impact of clone negatives. The modulating anchor is selected based on the distribution of negative samples without explicit training, allowing for progressive tuning and enhanced in-batch supervision. Extensive experiments on two benchmark datasets demonstrate the superiority of AdaCL. Furthermore, we extend AdaCL to weakly-supervised image-text matching by substituting human-annotated descriptions with automatically generated captions, increasing the number of potential clone negatives. AdaCL demonstrates robustness with the generated captions, alleviating the reliance on crowd-sourced annotations and laying a foundation for scalable vision-language contrastive learning.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Image-text matching Contrastive learning Vision and language Multimodal learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/a154c9f7fc18ce76b3660f631754a785b75be39a.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/6f1f73f42384e4562257861ef8f503696588ee5c.zip
Discovering Clone Negatives via Adaptive Contrastive Learning for Image-Text Matching
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission