GlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection
Zero-shot anomaly detection;Prompt learning;Object-agnostic glocal semantic prompt design;Deep-text prompt tuning;Glocal contrastive learning
Yes
No ethics review needed.
+ How to demonstrate the differences between global prompts $g_n$, $g_a$and local prompts $l_n$, $l_a$?
+ Does 'semantic prompt design' refer to Sec3.3 ? Why does it perform poorly on 'single', and even lead to a decrease in performance in table 4?
+ What causes the F3 to reduce image-level performance so drastically in medical applications in table 3? Could you provide a detailed explanation and analysis?
+ What' s the anchor prompts?
+ Some minor issues, such as line 316.
+  An object-agnostic global  and local prompts are supposed to learn normal and abnormal patterns.
+ Align visual features and text features by jointly optimizing global and local prompts through contrastive learning to enhance robustness.
+  The proposed method demonstrate excellent performance on multiple datasets.
This paper introduces a novel method for zero-shot anomaly detection. GlocalCLIP separates global and local prompts and optimize them together, enabling detection of general anomalies without object dependence. Through refined text prompts and a V-V attention layer for detailed image features, GlocalCLIP effectively captures both normal and abnormal patterns. Tested on 15 industrial and medical datasets, it outperforms current ZSAD methods.
This paper primarily focuses on improvements to AnomalyCLIP, particularly with regard to enhancements in text prompts. The paper has some unclear explanations that can make it difficult to understand.
Yes
No ethics review needed.
1. The structure of the proposed method appears to be almost identical to AnomalyCLIP, except for improvements in prompt design and contrastive learning. There is no fundamental change at the framework level.
2. The design of the V-V attention layer has already been experimented with and used in AnomalyCLIP.
3. I believe the incremental experiments in Table 3 should be conducted starting from AnomalyCLIP to demonstrate the effectiveness of the newly proposed innovations.
4. In Table 3, adding F4, which corresponds to GCL in Figure 4, does not seem to significantly improve the pixel-level metrics and even shows some decline. However, in Figure 4, adding GCL shows a clear difference in pixel-level performance compared to not adding GCL. Please explain the reason for this discrepancy.
The authors propose GlocalCLIP, which introduces separable global and local prompts through an object-agnostic glocal semantic prompt design and jointly optimizes them to address the zero-shot anomaly detection task. They also employ contrastive learning to enhance the learning of global and local visual features.
The authors propose GlocalCLIP, which introduces separable global and local prompts through an object-agnostic glocal semantic prompt design and jointly optimizes them to address the zero-shot anomaly detection task. They also employ contrastive learning to enhance the learning of global and local visual features. The effectiveness of the method is validated on 15 datasets.
See questions.
Yes
None
No ethics review needed.
Please refer to the Weaknesses section.
- The paper is clearly presented, including both text and images, making it easy to reproduce based on existing work.
- Quantitative and qualitative experiments on multiple datasets.
This work investigates the task of zero-shot anomaly detection and proposes a model named GlocalCLIP, which employs a dual-branch approach for both global and local modeling of image and text inputs based on CLIP. Results across multiple datasets demonstrate the effectiveness of the proposed method.
- The novelty of the method is limited compared to AnomalyCLIP. The framework still follows the standard Winclip framework and subsequent works. The authors' claim of the "first framework" is exaggerated, as prompt tuning and V-V attention have already been used in anomaly detection, particularly in AnomalyCLIP.
- The design of global and local branches is not novel. The authors should provide a more detailed explanation of what the two types of tokens model in zero-shot anomaly detection.
- The global loss does not use multi-stage feature fusion, unlike the local loss. Additionally, the novelty of the GCL loss is trivial, lacking any impressive design.
- The introduction resembles related work, and the motivation is not convincing.
- For Fig. 1, it is unclear whether the authors aim to highlight the novelty of the zero-shot anomaly detection task or the disadvantages of other settings. I disagree with the authors' claim in Sec. 2.4 that few-shot and multi-class models (e.g., UniAD) are unfriendly for practical applications.
- The improvement over AnomalyCLIP is not significant.
- Please report the differences in training overhead, computational costs, and efficiency comparisons of different methods.
Yes
No ethics review needed.
See Weaknesses
1. This paper focuses on a challenging and valuable field that is practical to the real world.
2. The authors conduct experiments across diverse datasets to support their claims.
This paper aims to adapt CLIP for the challenging task of zero-shot anomaly detection. The authors propose enabling object-agnostic learning to capture both global and local anomaly semantics. Experimental results seem to demonstrate the effectiveness of the proposed method.
1. This paper presents similar technological contributions and organization to AnomalyCLIP. However, the authors do not provide a comprehensive comparison and discussion with AnomalyCLIP. A detailed analysis in the introduction section is necessary to explicitly tell the main technological differences from AnomalyCLIP.
2. The manuscript lacks proper citations in several sections, notably in 3.2 (Prompt Design) and 3.5 (Training and Inference). The authors should carefully review and appropriately cite previous research to acknowledge foundational work in this field.
3. The illustration in the paper is unclear. For example, it is not evident what the term "anchor" refers to in Eq. 5, and the rationale for using V-V attention instead of other attention mechanisms, such as Q-Q or K-K attention, should be clearly explained.
@inproceedings{
anonymous2024glocalclip,
title={Glocal{CLIP}: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=mgjRSXGnF0},
note={under review}
}
Zero-shot anomaly detection (ZSAD) is crucial for detecting abnormal patterns in target datasets without using training samples, specifically in scenarios where there are distributional differences between the target domain and training data or where data scarcity arises because of restricted access. Although recently pretrained vision-language models demonstrate strong zero-shot performance across various visual tasks, they focus on learning class semantics, which makes their direct application to ZSAD challenging. To address this scenario, we propose GlocalCLIP, which uniquely separates global and local prompts and jointly optimizes them. This approach enables the object-agnostic glocal semantic prompt design to effectively capture general normal and anomalous patterns without dependency on specific objects in the image. We refine the text prompts for more precise adjustments by utilizing deep-text prompt tuning in the text encoder. In the vision encoder, we apply a V-V attention layer to capture detailed local image features. Finally, we introduce glocal contrastive learning to improve the complementary learning of global and local prompts, effectively detecting abnormal patterns across various domains. The generalization performance of GlocalCLIP in ZSAD was demonstrated on 15 real-world datasets from both the industrial and medical domains, achieving superior performance compared to existing methods.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Zero-shot anomaly detection Prompt learning Object-agnostic glocal semantic prompt design Deep-text prompt tuning Glocal contrastive learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/1036d89b9b38ea2df84f7dd0a9069a6cdf1ef118.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
GlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission