Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection
Video Anomaly Detection;Weakly-supervised Learning
Yes
No ethics review needed.
1. Considering that the Gaussian splatter extracts the Gaussian kernels from the peak values of the abnormal scores and renders it onto the curve, how are the peaks detected or selected? Please give more detailed explanation about the method of peak selection, especially when the curve fluctuates frequently?
2. Considering that misclassifications may often occur in MIL, please provide some analysis of such examples, such as incorrectly detected peaks on normal snippets.
3. Please provide experimental results on the UbNormal dataset to fully demonstrate the effectiveness of the proposed method.
Innovation: This paper proposes a new framework for weakly-supervised video anomaly detection framework that combines Gaussian splatter loss and mixture-of-expert architecture. The idea has some novelty in the field of video anomaly detection.
Performance: Experimental results show that the proposed GS-MoE achieves better performances than existing SOTA methods on both UCF Crime and XD Violence datasets with significant improvements, especially in handling complex abnormal events.
Presentation: This paper gives a detailed description for each component of GS-MoE, including the calculation of Gaussian splatter loss and the design of the mixture-of-expert architecture. This makes the proposed idea not difficult to be comprehended.
Experiments: This paper conducts extensive experiments on two widely-used datasets and compares their proposed method with other SOTA methods. Additionally, ablation studies and category performance analysis are also given to validate the effectiveness of each component.
This paper proposes the Gaussian Splatting-guided Mixture of Experts (GS-MoE) for weakly-supervised video anomaly detection. This method enhances the capability of class-expert models to understand the anomalous events in videos by utilizing the temporal Gaussian splatter loss, and captures the complex relationships between different anomalous patterns through a mixture-of-expert architecture. The experimental results show that this method has achieved significant performance improvements on the UCF-Crime and XD-Violence datasets. This paper proposes a promising new method for weakly-supervised video anomaly detection, and its effectiveness is proved by their experiments.
My concern about this paper mainly focuses on the computational complexity due to the use of a mixture-of-expert architecture, where the authors assign an expert for each anomaly. It seems that the proposed approach may consume a significant amount of resources. I suggest the authors discuss on the inference speed of the proposed model (such as FPS) as well in Table 1, so that the advantages and disadvantages of the model can be better illustrated.
Yes
No ethics review needed.
See the weaknesses.
1. The paper is well-written and clearly structured, making the ideas easy to follow.

2. The proposed GS-MoE framework is well-motivated, effectively addressing key limitations in weakly-supervised video anomaly detection. It introduces Temporal Gaussian Splatting (TGS) to reduce over-dependency on the most abnormal snippets, allowing the model to capture subtle temporal patterns across a broader range of anomaly cues. The Mixture of Experts (MoE) architecture further enhances performance by learning category-specific, fine-grained representations and connecting these with coarse anomaly cues, resulting in a more compact and accurate anomaly representation.

3. The approach achieves state-of-the-art results on UCF-Crime and XD-Violence, setting new benchmarks and demonstrating its effectiveness across various metrics.
This paper introduces GS-MoE, a novel framework for weakly-supervised video anomaly detection that combines a Mixture of Experts (MoE) architecture with Temporal Gaussian Splatting (TGS). TGS uses Gaussian kernels to capture broader temporal dependencies, enhancing the model’s ability to detect subtle and complex anomalies. The MoE architecture includes specialized expert models for different anomaly types, coordinated by a gating mechanism for fine-grained detection. Experimental results on UCF-Crime and XD-Violence datasets demonstrate state-of-the-art performance.
1. In Table 3, there seems to be a labeling error. The column labeled "With skip connect" should likely be labeled "With task-aware features"
Yes
No ethics review needed.
N/A
Pros: 
1. Usage of Gaussian kernels extracted from the estimated abnormal scores to generate complete representation 
2. Splatting the kernels along the temporal dimension for modeling anomalous temporal dependencies 
3. Dedicated class-expert models focus on individual anomaly types 
4. Improvements on benchmarks.
This paper proposes a Gaussian Splatting-guided Mixture of Experts (GS-MoE), leveraging a set of experts trained with a temporal Gaussian splatting loss on specific classes of anomalous events and integrating their predictions via a mixture of expert models to capture complex 
relationships between different anomalous patterns. Temporal gaussian splatting reduces the dependencies on the most abnormal snippets.
Cons: 
1. Some inappropriate expressisons and lack in sufficient literature review 
2. Each expert is trained only on refined features belonging to its assigned class and to the 
normal class. Why different classes are pre-defined like this, what if different classes are 
coupled? Anomalies are unexpected, it is difficult to define classes. The mixture of experts 
assumes that different anomaly types can be isolated effectively. However, in real-world 
applications, anomalies might not always fit neatly into predefined classes or could be ambiguous. 
This assumption may restrict generalization to unseen or blended anomaly types, particularly in 
more dynamic or less structured environments.
3. The model relies heavily on the quality of extracted features from pre-trained I3D models. 
If the feature extraction model underperforms or isn’t well-suited to certain video types, the 
overall anomaly detection may suffer. The reliance on a fixed feature extractor may limit 
adaptability across datasets with different visual characteristics. 
4. The combination of Gaussian splatting with a mixture of experts might make it difficult to 
interpret the model’s decision-making process. Although the framework shows strong quantitative 
results, it may be challenging to explain how or why certain frames are classified as anomalous, 
which could be important for fields requiring clear explanations of detection results.
Yes
No ethics review needed.
1. How does the method perform on other datasets with different types of anomalies, and have they considered any strategies to improve generalization?
2. What are the computational costs associated with training and deploying the GS-MoE framework, and how do they compare to other state-of-the-art methods?
3. How does the temporal Gaussian splatting loss impact the learning process, and can the authors provide more insights into how it improves the model's ability to detect anomalies?
4. Are there any immediate plans to incorporate interpretability or explainability features into the current model to understand the decisions made by the different experts?
1. The paper proposes a novel approach to WSVAD by combining Gaussian Splatting with a Mixture of Experts (MoE) architecture, which is a creative solution to address the limitations of current models in handling complex anomalies.
2. The framework demonstrates significant improvements over previous state-of-the-art methods on benchmark datasets, which is a strong indicator of the effectiveness of the proposed method.
3. The paper provides an extensive set of experiments and ablation studies that validate the effectiveness of the proposed contributions and provide insights into the impact of each component of the framework.
The paper introduces a novel framework named Gaussian Splatting-guided Mixture of Experts (GS-MoE) for weakly-supervised video anomaly detection (WSVAD). The approach addresses the challenges of detecting complex, real-world anomalies in videos by leveraging a set of expert models trained with a temporal Gaussian splatting loss on specific classes of anomalous events. These experts' predictions are integrated via a mixture of expert models to capture complex relationships between different anomalous patterns. The framework is designed to leverage temporal consistency in weakly-labeled data, enabling more robust identification of subtle anomalies over time. Additionally, this paper achieves state-of-the-art performance on the UCF-Crime and XDViolence datasets.
1. The paper would benefit from a more detailed analysis of the specific contributions of each architectural choice, such as the Mixture-of-Experts (MoE) design and the Temporal Gaussian Splatting (TGS) mechanism, to the overall performance. While the results are impressive, a deeper exploration of how these components enhance the model's capabilities would strengthen the paper's technical depth and provide readers with a better understanding of the innovations' impact. 
2. The paper does not sufficiently address how the GS-MoE architecture scales with increasing model size or its adaptability to different image resolutions and datasets. Providing insights into the model's scalability and flexibility would be crucial for establishing its practical applicability and robustness across various settings and data. 
3. The paper could benefit from a more detailed comparison with other MoE architectures to highlight the unique aspects of the proposed approach. Such a comparison would help position the GS-MoE framework within the broader landscape of anomaly detection methods and underscore its innovative features. 
4. Although the paper presents numerous visualization generation results, it lacks a discussion on failure cases. Including examples where the model underperforms or fails to detect anomalies would offer a more balanced view of the framework's limitations and areas for future improvement.
Yes
No ethics review needed.
What criteria are used to assign or develop expert models for specific types of anomalies?
Are there potential limitations in using this method for scenarios where anomalies are highly similar to normal events?
Could the authors elaborate on the framework's scalability and feasibility for deployment in resource-constrained environments?
The paper introduces a novel framework, GS-MoE, that effectively combines expert models to address the diverse nature of anomalies in WSVAD.
The approach is well-supported by comprehensive experiments, showing strong results and achieving state-of-the-art performance.
This paper proposes a GS-MoE framework for WSVAD to address the challenges of anomaly diversity and weak supervision by using multiple expert models and a temporal Gaussian splatting loss.
The description of the Gaussian splatting loss lacks detail, making it difficult to understand the exact mechanism by which it improves temporal consistency.
There is limited explanation regarding the selection and training of individual expert models for different anomaly types, which may affect reproducibility.
There is insufficient analysis on the computational cost and efficiency of the proposed mixture of experts approach, especially for real-time applications.
More clarity is needed on the implications of using Gaussian kernels, particularly how they impact the performance and generalization of the model.
Yes
No ethics review needed.
Introduction:
1. The current issues of VAD in paragraph 2 of the introduction is not well illustrated and the readers may find confusing as the definition fo the issues is not clear.
2. The motivation of introducing Gaussian splatting is not well depicted in the introduction.
3. The information of Figure 1 is extremely, which doesn't convey the central idea of this paper sufficiently. It is suggested that the authors should combine Figure1, Figure2 and Figure3 and give the readers an overview of the motivation of main idea.

Methods:
1. The innovation of the proposed method is limited as the main backbone is largely based on UR-DMU.
2. Figure 4 should add illustrations of the meaning of the icons involved.
3. Algorithm 1 seems unnecessary to presented in separate section, as plain words can presented well.

Experiments:
1. Comparisons with state-of-the-art VAD methods in 2024 should be included in Table 1.
2. The performance of XD-Violence is relative low. The authors may justify why such phenomenon happens.
3. The introducing of multi-experts inevitably increasing the computational overhead, and may further lead to issue of overfitting. The authors should conduct analysis from these aspects. 

Typos:
The authors should correct their citation format using command \citet{} or \citep{}, according to the formatting instructions.
1. The formulation of the WSVAD task using Gaussian kernels allows for a more expressive and complete representation of anomalous events. 
2. The Mixture-of-Expert (MoE) architecture focuses on individual anomaly types through dedicated class-expert models. This specialization, combined with the gate model's ability to leverage similarities and diversities among different types, enhances the model's overall effectiveness and robustness in anomaly detection.
This paper presents the Gaussian Splatting-guided Mixture of Experts (GS-MoE) framework, which addresses the limitations of existing models in Video Anomaly Detection (VAD) by employing multiple expert models. Each expert is trained on specific classes of anomalies, allowing the framework to effectively encode the unique characteristics of different anomalous events. The GS-MoE framework utilizes a temporal Gaussian splatting loss, which leverages temporal consistency in weakly labeled data to enhance the identification of subtle anomalies over time. By guiding the expert networks to concentrate on segments with a higher likelihood of containing anomalies, the proposed method improves performance in the weakly supervised setting.
1. The motivation of the proposed method is unclear.
2. The innovation of the method is limited as the backbone is largely based on existing method and multi-expert, and gated model seems common.
3. The illustrations of Figure is poor, which don't covey the central idea of the proposed method.
Yes
No ethics review needed.
See Weakness.
1. This paper is well-written and easy to understand.
2. The application of Gaussian Splatting and MOE concepts to WSVAD is very interesting.
3. Experiments were conducted on two datasets, and the performance of the proposed method significantly surpassed that of previous methods, especially on the UCF-Crime dataset.
The authors address weakly-supervised video anomaly detection (WSVAD) by presenting a new framework termed "GS-MoE." One key aspect of their approach is mining peaks from abnormal scores and proposing "Temporal Gaussian Splatting" to generate dense pseudo-labels from these peaks for supervision. Another key point is the proposed structure of MOE, which captures the diversity of anomalies through multiple parallel Expert blocks and uses a Gate block to fuse the results of different experts. Both qualitative and quantitative results demonstrate the effectiveness of the proposed method.
1. The core idea of Temporal Gaussian Splatting is to use Gaussian distribution to extend and smooth sparse binary snippet-level pseudo label, which has been proposed in [1].  This greatly reduces the contribution of the author's work.
2. Since the peak detection process can significantly impact the quality of the rendered anomaly score, the final performance can be greatly affected by the initialization process, including the choice of task encoder and the hyperparameters used for peak detection. Conducting more ablation studies on these factors could demonstrate the robustness of the proposed method.
3. The author mentions that the MOE architecture learns class-specific representations. However, it is known that the differentiation between experts in MOE [2] of LLM is achieved through an additional loss function. I am curious about how different experts within the proposed framework learn to focus on different anomalies using only video-level labels.
4. The ablation experiments for some important hyperparameters, such as $\sigma_i$ in Equation (3) and the number of experts $N$, were missing from the paper.
5. There are noun consistency issues in the paper, such as the use of "TGS" in Algorithm 1 and "TSG" in other parts of the paper.

[1] Zhang H, Wang X, Xu X, et al. GlanceVAD: Exploring Glance Supervision for Label-efficient Video Anomaly Detection[J]. arXiv preprint arXiv:2403.06154, 2024.

[2] Puigcerver J, Riquelme C, Mustafa B, et al. From sparse to soft mixtures of experts[J]. arXiv preprint arXiv:2308.00951, 2023.
A Novel Approach to Weakly-Supervised Video Anomaly Detection
@inproceedings{
anonymous2024mixture,
title={Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=rrdNQZRHEm},
note={under review}
}
Video Anomaly Detection (VAD) has proved to be a challenging task due to the in-
herent variability of anomalous events and the scarcity of data available. Under the
common Weakly-Supervised VAD (WSVAD) paradigm, only a video-level label
is available during training, while the predictions are carried out at the frame-level.
Despite decent progress on simple anomalous events (such as explosions), more
complex real-world anomalies (such as shoplifting) remain challenging. There
are two main reasons for this: (I) current state-of-the-art models do not address
the diversity between anomalies during training and process diverse categories
of anomalies with a shared model, thereby ignoring the category-specific key at-
tributes; and (II) the lack of precise temporal information (i.e., weak-supervision)
limits the ability to learn how to capture complex abnormal attributes that can
blend with normal events, effectively allowing to use only the most abnormal snip-
pets of an anomaly. We hypothesize that these issues can be addressed by sharing
the task between multiple expert models that would increase the possibility of cor-
rectly encoding the singular characteristics of different anomalies. Furthermore,
multiple Gaussian kernels can guide the experts towards a more comprehensive
and complete representation of anomalous events, ensuring that each expert pre-
cisely distinguishes between normal and abnormal events at the frame-level. To
this end, we introduce Gaussian Splatting-guided Mixture of Experts (GS-MoE),
a novel approach that leverages a set of experts trained with a temporal Gaussian
splatting loss on specific classes of anomalous events and integrates their predic-
tions via a mixture of expert models to capture complex relationships between
different anomalous patterns. The introduction of temporal Gaussian splatting
loss allows the model to leverage temporal consistency in weakly-labeled data,
enabling more robust identification of subtle anomalies over time. The novel loss
function, designed to enhance weak supervision, further improves model perfor-
mance by guiding expert networks to focus on segments of data with a higher like-
lihood of containing anomalies. Experimental results on the UCF-Crime and XD-
Violence datasets demonstrate that our framework achieves SOTA performance,
scoring 91.58% AUC on UCF-Crime.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Video Anomaly Detection Weakly-supervised Learning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7a7872095c2c176d42dfaf9cce6d2af0805479dc.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/c44b6b299df4c94df5dd6f969e16846cac9a3072.pdf
Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission