MamBEV: Enabling State Space Models to Learn Birds-Eye-View Representations
BEV;Bird’s Eye View;State space model;causal attention;3d computer vision
Yes
No ethics review needed.
Please refer to section Weaknesses.
1. This paper presents a method for applying Mamba to BEV detection.
The main content of the paper is focused on enabling state space models (SSMs) to learn birds-eye-view (BEV) representations, specifically in the context of 3D representation learning tasks. 

The authors propose a novel method called MAMBEV, which incorporates SSMs and linear attention to address the challenges of capturing temporal and spatial relationships in multiview video and fusing distinct visual representations. 

They conduct experiments using the nuScenes dataset and achieve results comparable to the state of the art.
1. The paper has several writing issues:
* Many sentences are lengthy, which hinders comprehension. For example, line 188, 191, 201.
* Certain sections lack citations, such as on line 206 and 207.
* Some areas seem to be missing punctuation, possibly in line 225.

2. The experimental comparison is not entirely fair, as the pretrained model used in this study differs from other pretrained models, which may affect the results.

3. The overall improvement in the experimental results is quite limited, and the value of applying SSM to BEV detection is not clear.

4. There is a lack of comparison with more results, such as VideoBEV, which is also a long-sequence application but was not included in the comparison.

5. The paper mentions reducing computational load and improving efficiency. Therefore, the proposed method should provide relevant metrics related to training and testing, such as training time and model latency during testing.
Yes
No ethics review needed.
1. An important reason for using Mamba is to achieve lower computational cost; however, the paper only provides theoretical calculations without experimental validation, which is a crucial aspect.
2. In the bottom experiments of Table 1, why do BEVFormerV1-Small and BEVFormerV1-Base use different frames?
1. Lower computational cost.
2. Good performance, comparable to VideoBEV.
The paper presents MamBEV, a novel 3D object detection framework designed to improve BEV-based perception systems, particularly for autonomous driving. By introducing state-spatial separation and multi-axis multiview projection, MamBEV addresses the challenges in handling multi-camera BEV representation. The proposed method achieves promising results in complex driving scenarios, showcasing improvements in detection accuracy and computational efficiency.
1.The paper could have provided more ablation studies to dissect the contributions of individual components of MamBEV.
2. Limited discussion on potential limitations of the framework, such as scalability or adaptability to other domains beyond autonomous driving.
3. limit novelty. The BEVFormer architecture introduces Mamba.
Yes
No ethics review needed.
- only tested on nuScenes (like some of the other SOTA models, including DETR3D and BEVformerV2). Is there another suitable dataset that this could be tested on to better demonstrate generality?
- significant problem domain where much improvement remains possible

- very well explained technical approach

- great to see asymptotic complexity analysis of the various stages of the approach

- convincing evaluation on the nuScenes dataset

- experiments on increasing temporal information, SSM vs deformable attention, feature normalization, scaling, and feature traversal order are interesting and convincing.
The paper proposes a new approach to building bird's-eye-view (BEV) maps from multiple cameras. With an aim of mobile deployment, the paper investigates approaches that depart from the existing transformer-based push (from images to 3D) or pull (from 3d to images) methods. In particular, recent work on state space models (SSM) is an inspiration to replace transformers and their quadratic attention computational complexity. Of particular interest here is to leverage ideas from the Mamba-2 and Hydra SSMs and adapt them to the BEV problem. The new SSM-based BEV model is well described and well tested, including several ablation studies.

Two potential caveats noted below are that only one dataset is used (as seems to be the case with other SOTA approaches, so likely not the authors' fault), and the improvements over SOTA are small and sometimes mixed depending on which metric one looks at.
- table 1, the proposed model does not outperform SOTA in all metrics. This table needs to be described better. If NDS is the main metric, then what about Focal-DETR? Should explain more clearly which table rows are compared in the main text lines 395-396

- should include a figure with reconstruction samples, for qualitative evaluation by the reader.

- lines 221-222: nonsense sentence
Yes
No ethics review needed.
The main issues have been raised in Weaknesses section.
1. MamBEV proposes a design based on SSM, which exceeds the performance of existing Transformer-based structures in several metrics.
2. MamBEV provides sufficient experimental results to illustrate the effectiveness of the design.
The paper introduces the Mamba mechanism into 3D visual perception task to learn bird's-eye view (BEV) representations in order to address computational and memory efficiency problems, and designs a Spatial Cross Attention module for the task. The effectiveness of the design is demonstrated through extensive experiments and ablation studies, which would bring benefits to application scenarios such as autonomous driving and driver assistance systems.
1. Lack of visualization of results. I am quite sorry to say that I am not familiar with the nuScenes dataset, so could the authors provide some visual examples to show the advantages of MamBEV?
2. Lack of more detailed description of the effectiveness of the Spatial Cross Attention module. Since the authors mention in the Introduction that "we further demonstrate that our method can better capture longer dependencies in multiview video", whether this word "better" means the module proposed in the paper comparing to other SSM Attention designs, or Mamba-2 comparing to Mamba-1, or Mamba comparing to Transformer, could it be supported and presented through other ways in addition to the accuracy results on the specific dataset?
Yes
No ethics review needed.
Please refer to the above.
- Enabling Mamba to BEV representation seems to be interesting.
The paper presents MamBEV, a novel framework designed for 3D visual perception tasks like 3D detection from multi-camera images, which are crucial for autonomous driving and assistance systems. The framework leverages a Mamba-based approach to learn unified Bird’s Eye View (BEV) representations using linear spatio-temporal SSM-based attention, which improves computational and memory efficiency across multiple 3D perception tasks. The key is SSM-based cross-attention, which functions similarly to standard cross-attention, enabling BEV query representations to interact effectively with relevant image features. Experiments are conducted to demonstrate the effectiveness of the method. 

Pos:
- Enabling Mamba to BEV representation seems to be interesting.

Cons:
- The authors claim that proposed method supports multiple perceptual tasks but has no proof.
- The article keeps emphasizing computational efficiency, but without proofs.
- The core figures are crude and difficult to understand
- Insufficient experimentation and comparison methods
Cons:
- The authors claim that proposed method supports multiple perceptual tasks but has no proof.
- The article keeps emphasizing computational efficiency, but without proofs.
- The core figures are crude and difficult to understand
- Insufficient experimentation and comparison methods
@inproceedings{
anonymous2024mambev,
title={Mam{BEV}: Enabling State Space Models to Learn Birds-Eye-View Representations},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=MvEkN2ejZ1},
note={under review}
}
3D visual perception tasks, such as 3D detection from multi-camera images, are essential components of autonomous driving and assistance systems. However, designing computationally efficient methods remains a significant challenge. In this paper, we propose a Mamba-based framework called MamBEV, which learns unified Bird's Eye View (BEV) representations using linear spatio-temporal SSM-based attention. This approach supports multiple 3D perception tasks with significantly improved computational and memory efficiency. Furthermore, we introduce SSM based cross-attention, analogous to standard cross attention, where BEV query representations can interact with relevant image features. Extensive experiments demonstrate MamBEV's promising performance across diverse visual perception metrics, highlighting its advantages in input scaling efficiency compared to existing benchmark models.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
BEV Bird’s Eye View State space model causal attention 3d computer vision
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/a04a005c1cbbac01267fd7d58820348d2acbf3ca.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
MamBEV: Enabling State Space Models to Learn Birds-Eye-View Representations
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission