Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection
Contrastive Learning;Knowledge Transfer;Open-Vocabulary Object Detection
Yes
No ethics review needed.
Most concerns are listed on above boxes. Hungarian matching based on CLIP embeddings is central to your contrastive knowledge transfer scheme, aligning regional embeddings with the teacher’s visual-semantic space. However, given that CLIP embeddings may not fully capture object-specific regional nuances, why would this Hungarian matching approach be expected to significantly improve performance?
This work is easy to follow. Most of the used techniques are correct.
This paper introduces CCKT-Det++, an open-vocabulary object detection framework designed to detect novel object categories without relying on extensive additional supervision. The approach leverages semantic priors as guidance and a regional contrastive knowledge distillation loss to improve the model's detection capability for novel classes. CCKT-Det++ utilizes visual-semantic embeddings from both VLMs and MLLMs to dynamically transfer knowledge from both image and text encoders, aligning the detector’s latent space with meaningful, structured knowledge.
1. While CCKT-Det++ performs well by leveraging stronger teacher models, it is heavily reliant on the quality and alignment of these teacher models. If the teacher model has limitations or biases, these could propagate to CCKT-Det++, affecting performance and potentially introducing unintended biases.

2. Although the student model maintains moderate parameters, using stronger teacher models during training and inference can introduce significant computational costs. Scaling the teacher models effectively requires access to substantial computational resources, which could limit practical applications for researchers without high-performance infrastructure.

3. The approach relies on pseudo annotations, which are less accurate than human annotations. If the pseudo labels are of low quality, they could introduce noise into the training process, possibly harming model performance on more challenging or nuanced object classes.

4. While semantic priors improve performance on novel objects, there is a risk that the model might over-rely on these priors, leading to reduced flexibility in identifying objects that fall outside its learned semantic scope. This dependency may affect its generalization to truly unseen classes in new environments.

5. The method heavily relies on CLIP for both semantic priors and feature extraction, making it vulnerable to limitations inherent in the CLIP model. For instance, CLIP's biases in language and visual associations could impact detection accuracy and lead to incorrect classifications in culturally or contextually sensitive settings.
Yes
No ethics review needed.
1. Clearly state the motivation for this paper.
2. It is necessary to give a clear definition of semantic prior and explain its difference from caption and prompts.
3. Please explain why the performance of the base class is not shown in the experiment.
1. The frameworks are explicit and effectively illustrates the idea of the method.
2. This method has an improvement in the detection performance of novel classes.
In order to solve the problem of relying on a large amount of additional data for open vocabulary object detection, the authors propose CCKT-Det method. This method guides the learning of queries by pre-filtering and injecting semantic priors, and introduces regional contrast loss to improve the perception of queries to novel objects.
1. The problem of the authors to solve and the motivation are both ambiguous. The authors point out that some methods rely on a large amount of extra caption data, but many methods can achieve high performance without it, such as the quoted OV-DQUO.
2. The authors introduce semantic prior, but do not clearly define the concept.
3. Figure 3 shows that the semantic prior needs to go through the VLM's text encoder, what is the difference between the semantic prior and the caption? And what is the difference between semantic priors and the prompts that currently popular? Neither of these issues is addressed by the authors.
4. The authors do not present the detection performance of the base class in the experiment.
Yes
No ethics review needed.
1. Why does the Regional-Contrastive Loss in the ablation study cause drop of AP50 (from 32.6 to 31.7)?
2. There are relatively few comparative methods on the OV-LVIS benchmark.
1. The proposed method is quite reasonable and can effectively transfer the visual and language capabilities of VLMs.
2. The experiments on the OV-COCO benchmark are relatively comprehensive, showing a significant improvement in detecting novel categories.
3. The ablation experiments are relatively thorough and comprehensive.
This paper presents CCKT-Det, a method for open-vocabulary object detection (OVD) that eliminates the need for extra supervision like image-text pairs or pseudo-labels. It leverages cyclic and dynamic knowledge transfer between language queries and visual region features from pretrained vision-language models (VLMs). The approach uses semantic priors for guiding query learning and a regional contrastive loss to enhance detection of novel objects. CCKT-Det achieves significant performance gains on the COCO benchmark, showing scalability with stronger VLMs while maintaining moderate computational requirements.
1. The introduction to the task setup is limited, and there is a lack of a focused, clear description and visualization of the workflow details of CCKT-Det during training and testing. Figure 2 is somewhat confusing.
2. The AP performance of CCKT-Det on the OV-COCO benchmark is relatively poor, and its detection capability on base categories is not as strong as other methods. This may be the cost of improving detection capabilities for novel categories.
3. The experimental results did not report the AP for the base category.
4. There is no comparison or analysis of time complexity.
Yes
No ethics review needed.
None
* The paper introduces a novel framework, CCKT-Det, for open-vocabulary object detection (OVD) that does not rely on any additional supervision beyond the base training set and achieves start-of-the-results on the COCO benchmark.
﻿
* Comprehensive experiments are conducted on multiple benchmarks (OV-COCO, LVIS, Objects365), demonstrating the method's effectiveness. The results show consistent performance improvements as the strength of the teacher model increases.  The paper includes thorough ablation studies to validate the effectiveness of each component of the proposed method.

* The paper demonstrates that high-performance OVD can be achieved without the need for additional supervision, such as image-text pairs or pseudo annotations. This is a significant advancement, especially in scenarios where such data may not be readily available.
The paper introduces CCKT-Det, a novel framework for open-vocabulary object detection (OVD) that operates without any additional supervision beyond the base training set. The method leverages semantic priors and regional contrastive knowledge distillation to align the detector with the visual-semantic space of vision-language models (VLMs).  This paper presents semantic priors to boost the detector's capacity for novel object recognition and a unique loss function that heightens the detector's sensitivity to new objects by aligning region-level features. Based on these contributions, this work achieves start-of-the-art results on COCO benchmark.
* It is better to add a more detailed result comparison in Tab.1 and Tab.2. For example, show the AP_Base result for COCO and AP_c and AP_f results for LVIS.
* This method has shown promising results in detecting novel objects, but it seems not very good for detecting base objects.
Yes
No ethics review needed.
- In the paper, you mention, “In contrast to previous works, where object queries are static and fixed for each image after training, we dynamically inject text embeddings {ti} as semantic priors into object queries to form the language queries for each image.” However, similar ideas have been explored in earlier works, such as GroundingDINO. It would be appropriate to supplement your references by citing these related works and discuss their similarities and differences.
- In Table 4, the paper states that “While semantic guidance demonstrates effectiveness to a degree, its efficacy is constrained in the absence of regional contrastive training.” However, contrary to this claim, regional contrastive appears to have a detrimental effect on semantic guidance, as the ablation experiment did not demonstrate any significant effectiveness of regional contrastive training. This seems to contradict the statement in the paper, “As indicated in the third row of Table 4, the absence of this loss results in a performance decline of 5.4% AP50 on novel classes.” Based on the results, there appears to be no significant relationship between Regional-Contrastive Loss and Semantic Guiding, while it works in conjunction with Similarity Classification. This seems to be a clerical error.
The paper is well-structured, with solid methodological rigor. The authors present comprehensive experiments and ablation studies that demonstrate the effectiveness of their proposed method. The proposed CCKT-Det framework has high significance in advancing the state of open-vocabulary object detection. By eliminating the need for additional supervision and showing competitive performance with existing methods that do require extra data, the paper addresses a key limitation in the field.
The paper introduces Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection (CCKT-Det), a method designed to enhance the detection of novel objects in open-vocabulary settings without relying on additional supervision like image-text pairs or pseudo annotations. It leverages pretrained vision-language models (VLMs) to transfer knowledge effectively from base categories to novel ones. The key contributions include using semantic priors to guide object queries and a regional contrastive knowledge distillation mechanism to align the visual features between a student and teacher model. CCKT-Det achieves state-of-the-art performance on COCO and LVIS benchmarks, improving novel object detection without extra data, and scales efficiently with stronger VLMs while maintaining moderate computational overhead.
- Although you claim and emphasized CCKT-Det did not rely on the extra data, in contrary to the previous method, you use the extra MLLM to be a discriminator and generate the prior semantic guidance.
Our proposed framework constructs a cyclic and dynamic matching between semantic priors and visual region features extracted from VLMs, which forces the detector to closely align with the visual-semantic space of VLMs.
@inproceedings{
anonymous2024cyclic,
title={Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=JU9oHs7ivN},
note={under review}
}
In pursuit of detecting unstinted objects that extend beyond predefined categories, prior arts of open-vocabulary object detection (OVD) typically resort to pretrained vision-language models (VLMs) for base-to-novel category generalization. However, to mitigate the misalignment between upstream image-text pretraining and downstream region-level perception, additional supervisions are indispensable, e.g., image-text pairs or pseudo annotations generated via self-training strategies. In this work, we propose CCKT-Det trained without any extra supervision. The proposed framework constructs a cyclic and dynamic knowledge transfer from language queries and visual region features extracted from VLMs, which forces the detector to closely align with the visual-semantic space of VLMs. Specifically, 1) we prefilter and inject semantic priors to guide the learning of queries, and 2) introduce a regional contrastive loss to improve the awareness of queries on novel objects. CCKT-Det can consistently improve performance as the scale of VLMs increases, all while requiring the detector at a moderate level of computation overhead. Comprehensive experimental results demonstrate that our method achieves performance gain of +2.9% and +10.2% AP_{50} over previous state-of-the-arts on the challenging COCO benchmark, both without and with a stronger teacher model.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Contrastive Learning Knowledge Transfer Open-Vocabulary Object Detection
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/3fd28ea51b347150eef8dec9402c937bac11e00d.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detection
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission