Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking
Synchronized sequence modeling; multiple object tracking
Yes
No ethics review needed.
- Just to confirm, does the state space model operate on the coordinate domain (bounding boxes) or the visual domain (pixel values)? If it does operate on the coordinate domain, how can the auto-regressed output boxes be refined to fit the subjects without looking at visual features?
- Overall, the experiments are extensive with different alterations in modeling provided in the main paper and appendix.
The paper introduces SambaMOTR for multiple object tracking in videos. This method aims to improve tracking by utilizing temporal information about object movements to model historical object data, and occlusions, thereby modeling long-range dependencies, implicit motion, and appearance between tracklets.
- Visual comparisons show that the Samba autoregressive model performs better than memory/motion models should be provided to substantiate the claims.
- The interpretations of system $\mathbf{A}$ and control $\mathbf{B}$ matrices in the context of object tracking are unclear. Providing clearer definitions of notations would be helpful.
- I understand the intention to separate two paradigms tracking-by-detection (TBD) and tracking-by-propagation (TBP), but the difference is subtle as the TBP still needs bounding boxes to initiate the states, and needs a detector to recognize newly appearing objects. Then, propagation alone is simply not practical in this problem of multiple object tracking.
- Then it should be fair to compare with TBD methods, and the performance still falls short behind classic motion models as reported in Table A.
- While I value the effort in development, the innovation in the method appears to be limited, as it essentially represents a straightforward combination of MOTR and Mamba and the motivation is not really compelling.
Yes
No ethics review needed.
(1) Would it be possible to include a comparison of SambaMOTR on top of YOLOX-X to have an apple-to-apples comparison with prior tracking-by-detections works?
(1) The presented method achieves strong performance.

(2) the reason to integrate SSMs into a tracking-by-propagation framework is well motivated.

(3) The ablation studies conducted verified the introduced modifications.

(4) The paper is well written; the provided illustrative examples showcase strong results.
The paper introduces the novel SAMBA model architecture for multi-object tracking. It takes a unique approach to tracking in a tracking-by-propagation framework, building on top of selective state-space models captured by Mamba models. Additional self-attention is used to model dependency between tracks. The method is evaluated on three tracking datasets, and it demonstrates a new state-of-the-art performance.
(1) There is a lack of qualitative comparison between prior works and the proposed method. It is helpful to see qualitative differences between methods' outputs to highlight what improvements in the models' output are contributing to the improvement in the performance. 

(2) On L137, it is stated that  SambaMOTR has "the same GPU memory requirements". However, the paper lacks any measurements to back up this claim. Overall, there is emphasis placed on efficiency in the paper, so it would be good to include some measurements and comparisons to show these points. For example, including peak memory requirements and FLOPS would illustrate this point. 

(3) While the method shows strong results, chiefly due to architectural modifications, it is worth asking whether efficiency gains enable larger (higher capacity) models, which lead to improvements or whether the SSMs formulation provides better biases for the tracking problem. While it might be difficult to disentangle such factors, the paper could help guide future research in this area by reporting some (albeit not great) proxies, such as learnable parameter counts, etc.
Yes
No ethics review needed.
see Weaknesses.
1. SambaMOTR introduces the novel linear-time set-of-sequences model designed to jointly process multiple tracklets by synchronizing the multiple selective state-spaces used to model each tracklet.
2. SambaMOTR achieves good tracking results on DanceTrack, BFT, and SportsMOT datasets.
The paper introduces a tracking-by-propagation framework, SambaMOTR, which wanna tackles the question of how to model long-range dependencies within tracklets, interdependencies among tracklets, and the associated temporal occlusions. The core function of the  SambaMOTR is the Samba module, a novel linear-time set-of-sequences model designed to jointly process multiple tracklets by synchronizing the multiple selective state-spaces used to model each tracklet.  SambaMOTR is evaluated on DanceTrack, BFT, and SportsMOT datasets and achieves the state-of-the-art performance.
1. The authors are advised to analyze the computational complexity.
2. The authors are advised to analyze why Samba is suitable for trajectory modeling and compare it with xLSTM, other SSMs, and RRNs.
3. The authors are advised to provide some tracking cases of SambaMOTR to demonstrate its superiority in trajectory modeling.
SambaMOTR is a tracking-by-propagation method that (i) leverages long-range dependencies and (ii) synchronizes memory across object trajectories to better estimate future track queries and handle occlusions.
@inproceedings{
anonymous2024samba,
title={Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=OeBY9XqiTz},
note={under review}
}
Multiple object tracking in complex scenarios - such as coordinated dance performances, team sports, or dynamic animal groups - presents unique challenges. In these settings, objects frequently move in coordinated patterns, occlude each other, and exhibit long-term dependencies in their trajectories. However, it remains a key open research question on how to model long-range dependencies within tracklets, interdependencies among tracklets, and the associated temporal occlusions. To this end, we introduce Samba, a novel linear-time set-of-sequences model designed to jointly process multiple tracklets by synchronizing the multiple selective state-spaces used to model each tracklet. Samba autoregressively predicts the future track query for each sequence while maintaining synchronized long-term memory representations across tracklets. By integrating Samba into a tracking-by-propagation framework, we propose SambaMOTR, the first tracker effectively addressing the aforementioned issues, including long-range dependencies, tracklet interdependencies, and temporal occlusions. Additionally, we introduce an effective technique for dealing with uncertain observations (MaskObs) and an efficient training recipe to scale SambaMOTR to longer sequences. By modeling long-range dependencies and interactions among tracked objects, SambaMOTR implicitly learns to track objects accurately through occlusions without any hand-crafted heuristics. Our approach significantly surpasses prior state-of-the-art on the DanceTrack, BFT, and SportsMOT datasets.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Synchronized sequence modeling; multiple object tracking
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/7518b9d4c437e572f7afe0ae26e9a8be945459b1.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission