Unleashing the Potential of ConvNets for Query-based Detection and Segmentation
Object Detection;Segmentation;ConvNet;Segment Anything
Yes
No Ethics Concerns
No ethics review needed.
Please help to address the clarity issues.
- The DECO method requires only standard convolutions, making it computationally efficient and more compatible with low-end AI chips compared to DETR-based methods.

- The designs of self-interconv and cross-interconv are extremely simple and effective, indicating that the paper captures the essence of DETR and presents an elegant pure-conv replacement.
The paper proposes an InterConv mechanism to facilitate interaction between object queries and image features via a convolutional layer, resulting in the Detection ConvNet (DECO). DECO is composed of a backbone and a convolutional encoder-decoder architecture. Results on COCO detection demonstrate that DECO achieves a decent accuracy-speed trade-off.
- Clarity: In the method section, the Fusion operation is crucial; however, the paper fails to discuss it adequately. In the experiment section, the paper presents different fusion methods and query map upsampling strategies, but they remain unclear. These important details should be clearly stated and explained in the method section.

- The paper is missing some important references, such as the large kernel CNN paper and the SparseInst paper, which is also a pure-conv query-based object detection/segmentation method.
Yes
No ethics review needed.
1. Can the denoising training technique in DN-DETR and DINO be introduced into convolutional-based DECO?
2. To further improve the performance, it may be practical to introduce the deformable convolution (v1-v4) into the encoder and decoder. What about the efficiency?
1. The proposed InterConv consisting of pure convolutional layers can help the encoder and decoder capture long-range details without attention mechanisms and also help the decoder interact with object queries.
2. Without many complicated designs, DECO achieves comparative performance and efficiency over DETR.
3. DECO can be further improved to DECO+ by introducing the multi-scale design from Deformable-DETR.
This paper proposes a query-based end-to-end Encoder-Decoder architecture, i.e., DECO with pure CNN modules for object detection, built upon a mechanism dubbed InterConv for the interaction between object queries and image features. Experimental results show that the proposed method without complicated components achieves competitive performance and efficiency over the DETR series.
1. On Lines 074-075, within the scope of object detection, the statement ``whose weights and inputs are both generated during runtime'' is unclear, as the only ones dynamically generated are intermediate variables (including attention weights).
2. The performance can be further improved using more convolutional-based techniques like deformable convolutions.
Yes
N/A
No ethics review needed.
1. I'm concerned about the initialization of the object queries. Could the object queries be random initialized variables for different images?
2. In Sec. 4.4, does this paper combine DECO with TinySAM?
1. This paper presents InterConv which mimics attention mechanism with convolutional layers, and formulates Self-Interaction Module and Cross-Interaction Module (CIM) as self-attention and cross-attention.
2. This paper explores a convolution-based DETR framework with the proposed InterConv.
3. The proposed DECO obtains comparable performance compared to DETR.
The paper presents a new approach to object detection and segmentation using Convolutional Neural Networks (ConvNets), challenging the dominance of transformer-based architectures in these fields. This paper introduces InterConv, a mechanism that mimics the interaction between object queries and image features, traditionally handled by attention mechanisms in transformers, but using only convolutional layers. This paper presents Detection ConvNet (DECO), an end-to-end object detection framework that replaces transformers with ConvNets. The proposed DECO obtains comparable results compared to transformer-based methods.
1. Although this paper presents the convolution-version DETR which replaces attentions with convolutions, the overall performance is inferior to recent transformer-based methods, such as DINO, Co-DETR, and Stable-DINO[1], showing the limitations of using convolutional architectures.
2. In Tab. 2, it shows that DETR achieves better performance than DECO on both R50 and ConvNeXt-T while the improvements in inference speeds are not significant. In addition, DETR can be further optimized for acceleration therefore obtaining better inference speed and accuracy. Hence, the effectiveness of the proposed method in this paper is rather limited.
3. This paper lacks experimental comparisons with many recent works, such as Stable DINO[1], RT-DETRv2[2], DDQ[3], and SpeedDETR[4], in terms of both accuracy and speed.
4. It's hard to transfer the well-established techniques of recent DETR variants, such as denoising queries and deformable attention, into DECO for better performance.
5. The proposed object queries require prior knowledge to determine the best shape/layout, for example, 30x10 queries for the COCO dataset. I'm very concerned about whether the pre-trained detector DECO can perform well on the other datasets with different aspect ratios, such as datasets with 1:3 aspect ratios.
6. The details of extending DECO to segmentation tasks are unclear.



References\
[1] Liu et al. Stable-DINO: Detection Transformer with Stable Matching. ICCV 2023.\
[2] Lv et al. RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer. arXiv 2024.\
[3] Zhang et al. Dense Distinct Query for End-to-End Object Detection. CVPR 2023.\
[4] Dong et al. SpeedDETR: Speed-aware Transformers for End-to-end Object Detection. ICML 2023.
Yes
No ethics review needed.
NA
+ The paper is well-written and easy to follow.
+ Experiments are strong, demonstrating the effectivenss of CNN-based model.
The authors introduced a CNN-based end-to-end detection architecture, DECO. DECO contains an encoder and a decoder with pure convolutions nad is easy to implement. The experimental results demonstrate the effectiveness.
+  My main concern is the use of cross-scale feature-fusion module which is proposed in RT-DETR, which weakens the contribution. It would make the paper more strong if the authors could further discuss the difference.
+ There are many recent studies speeding up transformers, it would be nice to discuss more on using convolutions. Claims in L073-L074 are not convincing to me.
@inproceedings{
anonymous2024unleashing,
title={Unleashing the Potential of ConvNets for Query-based Detection and Segmentation},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=TWRhLAN5rz},
note={under review}
}
Transformer and its variants have shown great potential for various vision tasks in recent years, including image classification, object detection and segmentation. 
Meanwhile, recent studies also reveal that with proper architecture design, convolution networks (ConvNets) also achieve competitive performance with transformers, \eg, ConvNeXt. 
However, no prior methods have explored to utilize pure convolution to build a Transformer-style Decoder module, which is essential for Encoder-Decoder architecture like Detection Transformer (DETR).
To this end, in this paper we explore whether we could build query-based  detection and segmentation framework with ConvNets instead of sophisticated transformer architecture.
We propose a novel mechanism dubbed InterConv to perform interaction between object queries and image features via convolutional layers. 
Equipped with the proposed InterConv, we build Detection ConvNet (DECO), which is composed of a backbone and convolutional encoder-decoder architecture. We compare the proposed DECO against prior detectors on the challenging COCO benchmark.
Despite its simplicity, our DECO achieves competitive performance in terms of detection accuracy and running speed. Specifically, 
with the ResNet-18 and ResNet-50 backbone, our DECO achieves $40.5\%$ and $47.8\%$ AP with $66$ and $34$ FPS, respectively. The proposed method is also evaluated on the segment anything task, demonstrating similar performance and higher efficiency.
We hope the proposed method brings another perspective for designing architectures for vision tasks.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Object Detection Segmentation ConvNet Segment Anything
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/153cbe0eab71aea2a2f5534639e319f3985ad57f.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Unleashing the Potential of ConvNets for Query-based Detection and Segmentation
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission