MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection
Test-Time Adaptation;3D Object Detection
Yes
No ethics review needed.
Most questions are mentioned in the weakness section above.
Whether MOS can be generalized to other types of data, such as inputs combining LiDAR point clouds and camera images?
How checkpoints are selected and what "unique insights" would improve understanding of the model selection process?
How does the computational cost of calculating the synergy weights scale with the size of the model bank and batch size?
Also, I'm curious about if there's any evaluation of the impact of the proposed method on adaptation scenarios involving different types of LiDAR sensors or point cloud resolutions.
I appreciate the response to these questions.
The paper introduces a novel weighted ensemble of historical checkpoints approach, that helps mitigate catastrophic forgetting while adapting models to unseen domain shifts. This method achieves SOTA performance across multiple datasets and scenarios. 
The proposed method also effectively reduces memory usage by dynamically updating the model bank, retaining only key models with high synergy weights, thus maintaining performance with low resource cost
This paper introduces a novel approach for test-time adaptation in LiDAR-based 3D object detection: Model Synergy (MOS). The proposed method dynamically selects and assembles historical checkpoints to create a composite "super model" that adapts to domain shifts, including cross-dataset and corruption shifts. The assembly of checkpoints is controlled by Synergy Weights (SW) module, which minimizes redundancy by leveraging output-level and feature-level similarities. The method addresses practical scenarios like cross-corruption shifts, where both dataset and environmental discrepancies occur simultaneously. Experimental results across multiple datasets, including KITTI, Waymo, and KITTI-C, demonstrate that MOS outperforms SOTA test-time adaptation methods, achieving considerable improvements in both standard and complex scenarios.
The framework proposed by this paper is generally novel and effective, however, I still have some questions about the approach:
1. The paper demonstrates strong results (outperforms the SOTA methods) for LiDAR-based 3D object detection but does not extend its findings to other modalities or sensors. I'm interested in whether MOS can generalize to other types of data, such as inputs combining LiDAR point clouds and camera images.
2. While the paper contains empirical results for the chosen similarity metrics, more theoretical analysis for determining the synergy weights could be more thoroughly discussed.
3. It would be better if the authors could provide more clarity on how checkpoints are selected and what "unique insights" would improve understanding of the model selection process, I may not quite follow what's the insights here.
4. The rank-based feature-level and output-level similarity functions calculate the inverse Gram matrix, this method is novel but seems to be computational heavily. The paper provides some analysis of the MOS method in the last section, but it'll be great to have more discussion on how these similarity metrics scale with larger datasets or different types of models.
Yes
No ethics review needed.
To maintain feature diversity, the method assigns higher weights to checkpoints with unique insights. This may lead to more false positive results; how does the method address this issue?

Does improving accuracy at the cost of time and computational resources go against the original intent of test-time adaptation?
-The paper focuses on exploring test-time adaptation for 3D object detection and proposes a natural and effective model synergy strategy that dynamically assigns weights to historical checkpoints and aggregates knowledge to tackle task. 
-The paper analyzes and discusses the effectiveness of the synergy weights and makes task-specific modifications. 
-Experiments verifies the method's effectiveness under various conditions, outperforming comparison methods.
-The paper is well-writing and easy to follow.
The paper explores the test-time adaptation for LiDAR-based 3D object detection and proposes a model synergy approach that leverages long-term knowledge from historical checkpoints to tackle this task. Experiments are shown to verify the approach by tested against existing test-time adaptation strategies across three datasets.
The theory on which the method is based has been considered in prior works. The paper makes minor modifications specific to the 3D object detection task.
A few minor clarifications are required in approach introduction， I have elaborated in the following section
Yes
No ethics review needed.
- From Figure 7, the reviewer notes that both the time and memory costs of MOS are increased. Do the authors have any ideas for optimizing this additional cost?
- The reviewer wonders about the average time cost of conducting one experiment.
- The concept of weighting checkpoints based on newly acquired knowledge is intriguing.
- The cross-dataset &environment setting can positively impact this field.
- This work is well-supported by experiments and ablation studies.
This work proposes MOS, a framework that dynamically weights a sequence of models to create a more robust model based on the richness of new knowledge. MOS is evaluated in a test-time adaptation setting for LiDAR-based 3D object detection tasks. Experiments conducted in cross-dataset, cross-environment and cross-dataset&environment settings demonstrate the effectiveness of MOS.
- Additional visualization of detection results is needed, particularly a comparison between methods that do not use new-knowledge-based model aggregation and MOS.
- From Table 9, the reviewer observes that the pedestrian detection results are unsatisfactory, and from Table 8, the reviewer notes that the cyclist detection results could be improved. This could benefit from adding 2D supervision, as rich contextual and color information in 2D images can facilitate the detection of small objects. A similar approach is suggested in recent work: Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scene (ECCV '24). The authors are encouraged to include relevant references and discussions.
@inproceedings{
anonymous2024mos,
title={{MOS}: Model Synergy for Test-Time Adaptation on Li{DAR}-Based 3D Object Detection},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=Y6aHdDNQYD},
note={under review}
}
LiDAR-based 3D object detection is crucial for various applications but often experiences performance degradation in real-world deployments due to domain shifts. While most studies focus on cross-dataset shifts, such as changes in environments and object geometries, practical corruptions from sensor variations and weather conditions remain underexplored. In this work, we propose a novel online test-time adaptation framework for 3D detectors that effectively tackles these shifts, including a challenging $\textit{cross-corruption}$ scenario where cross-dataset shifts and corruptions co-occur. By leveraging long-term knowledge from previous test batches, our approach mitigates catastrophic forgetting and adapts effectively to diverse shifts. Specifically, we propose a Model Synergy (MOS) strategy that dynamically selects historical checkpoints with diverse knowledge and assembles them to best accommodate the current test batch. This assembly is directed by our proposed Synergy Weights (SW), which perform a weighted averaging of the selected checkpoints, minimizing redundancy in the composite model. The SWs are computed by evaluating the similarity of predicted bounding boxes on the test data and the independence of features between checkpoint pairs in the model bank. To maintain an efficient and informative model bank, we discard checkpoints with the lowest average SW scores, replacing them with newly updated models. Our method was rigorously tested against existing test-time adaptation strategies across three datasets and eight types of corruptions, demonstrating superior adaptability to dynamic scenes and conditions. Notably, it achieved a 67.3% improvement in a challenging cross-corruption scenario, offering a more comprehensive benchmark for adaptation. The source code is provided in the supplementary materials.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Test-Time Adaptation 3D Object Detection
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ad6f586bbb5990a328abf89b6a7122da31aa0f02.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/0b6062c14f43c6d2a3c7d55cf9c99ef4acb2d8c7.zip
MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission