CoMotion: Concurrent Multi-person 3D Motion
human pose estimation;3d human pose;tracking
Yes
No ethics review needed.
I really appreciate authors' hard work, which gives fast and accurate results on the benchmarks. However, I am doubtful of this paper's novelty and insight and I give borderline reject accordingly.
1. This paper considers a new pipeline to address multi-person video-based tracking and pose estimation. 
2. The paper is well-written and easy to understand. The qualitative results clearly show the advance of the proposed method. 
3. This method is very fast compared to other methods and achieve good results in several benchmarks.
This paper addresses a challenging problem of multi-person 3D pose tracking. Compared to tracking-by-detection pipelines, this paper extends the tracking-by-attention pipelines from Multi-Object Tracking to Multi-person pose tracking, which detect new humans and update existing tracks by learned tokens. Due to the lack of complete training data, this paper explores the mixed datasets with different forms of supervision. The experimental results demonstrate superior performance over other state-of-the-art methods on several benchmarks.
1. This paper is more like a technical report rather than research paper. The contributions are hard to recognise and not clearly highlighted. This makes this paper less insightful. 
2. To make the contributions clearer, i would recommend giving some preliminaries about the tracking-by-attention pipelines applied in other applications, e.g., MeMOTR, to distinguish what’s new in your method and what has been done.
3. One important claim in your paper in CoMotion can utilise subtle pixel cues in Line52-53, which is not elaborated is supported in the following text and experiments.
4. The compared methods in tab.1 seems not very recent, what’s the comparison between CoMotion and more advanced MOT methods?
5. Some minor issues: 
- Metrics can be introduced. If the space is not enough, you can include them in the appendix.
- I would recommend give short descriptions about ablation studies instead of checking them directly in appendix.
Yes
No ethics review needed.
1. The videos show unstable pose estimation for static humans, which should be addressed better in the pose tracking method than in the single-frame human pose estimation method. What is the cause of such instability, as the model has information about previous estimations and a learned hidden state? 
2. The authors mentioned using a 'strict' threshold in evaluating pose track metrics. Could the authors elaborate more on the choice of threshold and its relationship with the tracking method's precision and recalls? 
3. Will the proposed method be open-sourced? There is only limited information on the model architecture and hyper-parameters. It will be hard to reproduce the proposed method if the source code and checkpoint are not available. 
4. The proposed method has worse results compared to CLIFF in Table 3. What's the possible cause of it? 
5. Dense object tracking has been much improved by papers like CoTracker. How will the proposed method perform when compared to the method Cotracker? It will be interesting to see the performance difference between a 3D-aware object-specific method performed compared to a generic method.
1. The proposed method estimates the tracking and pose simultaneously, which could take advantage of prior information on humans.
2. The design choice of hidden state and GRU is sound. Keeping tracking alive seems to be a valid approach to address the missing people in a shorter period. 
3. The evaluation results on PoseTrack benchmarks are impressive, validating the proposed method's design choice.
This paper addresses the problem of 3D human pose and shape estimation and cross-frame tracking. The proposed method estimates 3D poses for all the persons in a frame and then updates the tracking poses and bounding boxes. A novel pose update module with GRU is introduced to update the tracking results and store the hidden information not present in the SMPL parameters. The proposed method is benchmarked on both bounding box tracking and pose estimation tasks and demonstrated better performance than the previous method. The comparisons on per-frame runtime also demonstrated its much more efficient design compared to the previous method.
Overall, the paper is of good quality, but there are still some weaknesses that the reviewer would like to discuss here and in the following questions. 
1. The implementation details and model architecture are unclear. The model architecture of both the pose estimation module and the pose update module is unclear. It's also unclear how the detection of bounding boxes is achieved. 
2. Although the hidden state and GRU are sound approaches, the experiments do not rigorously evaluate their effectiveness. 
3. The proposed model takes advantage of multiple training datasets, although it's not uncommon in the community of human pose estimation, but part of the improvements in the tracking results might be attributed to this.
Yes
No ethics review needed.
L318: what is the “off-the-shelf MAE”?

I wonder if the authors have thought about utilizing a generative model as pose prior, ie. Following ScoreHMR. The system could then be understood as a Recursive Bayesian Filter, which would provide a nice mathematical foundation for the tracking algorithm.
The method solve a highly relevant problem, tracking multiple people from a monocular camera. The method performs well on tracking and pose estimation benchmarks and the authors identified and fixed a bug in existing benchmarks, which is very helpful to the research community.
CoMotion introduces a complex system for multiple person tracking from a monocular camera, based on the tracking-by-attention paradigm. To overcome the challenge of limited data for pose estimation the authors train the model in multiple stages and add various datasets to gradually teach the model to track 3D persons across time.
The method makes use of various datasets and human pose modalities, such as SMPL, pseudo-SMPL and COCO. Could the authors describe how they handle different pose skeleton modalities, i.e. how do they train their SMPL pose estimation model with COCO skeleton supervision?

LL292-299: it is interesting that the authors do not add tracks during training: have the authors conducted an analysis of the training data to determine how many tracks appear / disappear on average? I would assume that this could potentially be very detrimental to the learning, as the model might learn to “ignore” certain people (i.e. if they were occluded before, or if they are at the edge of the image). Have the authors done any analysis of those kind of edge cases?

What do the authors mean by “[we] do not expect the shape of the mesh to correspond to the person’s physical body” (LL207-208)? The shape does not just model the mesh surface but also influences the bone lengths, so the skeleton is affected by it as well. Could the authors clarify this?

What do the authors mean by “the intrinsic matrix [does not] necessarily reflects the actual calibration of the camera“? Especially, as the authors claim in the next sentence that their method is robust to any provided intrinsic camera! Such a claim should be ablated, i.e. compare the system predictions with K provided and without K provided.

Can the authors elaborate on what they mean by “strict” in Table 2?

Figure 2 is not self-containing as it does not describe all elements of the image, i.e.e method part (A) is missing.

Typo in L205-206: the parameters $t$, $\beta$ and $\theta$ are missing from the SMPL parameter discretion. 

LL324-325: the authors should provide more details on how they apply the regularization for the beta optimization.

— suggestions — (will not influence my rating)

The authors could utilize the CHI3D [a] as an additional benchmark as it comes with well-tracked 3D poses (SMPL-X) at higher quality than 3DPW and with more challenging human-human interactions.


[a] Fieraru, Mihai, et al. "Three-dimensional reconstruction of human interactions." CVPR 2020.
Yes
No ethics review needed.
See weakness part.
1. Investigating the tracking-by-attension way of mutl-person 3D human motion tracking in crowded scenes is very challenging and interesting. The results of this paper is quite encouraging and exciting.   
2. Updating the tracklet state with assistance of bouding box detection is reasonable. Design of 3D human pose update module is great and seems to be very effective. Subtle 3-stage training process is brilliant.  
3. The experiment details are sincerely and clearly explained.
This paper describes a method to simultaneously track and regress coherent 3D human motion from videos with occlusion scenes. The key innovation is to update the 3D pose of multiple tracking targets from sequential images in a frame-by-frame manner. The baseline method for  comparison is 4DHuman, which tracks people via artificially associating the 3D human pose, location & appearance based on pre-defined matching rules. While the proposed method follows a tracking-by-attention way, which detects objects and updates existing tracklets via cross-attention between image and tracklet tokens. Query tokens of existing tracklets are firstly updated with current image tokens. The updated 3D human poses are matched with 2D bouding box detection of current frame. The unmatched detection will be initialized with new tracks and get updated to generate new 3D pose. Evalutation results on PoseTrack21 testified the effectiveness of the prospoed method. Compared with 4DHumans, the MOTA, IDF1, and IDR get significantly improved.
1. The tracking experiment is mostly performed on PoseTrack dataset only. Maybe some challenging occlusion scenes in 3D benchmarks, such as MuPoTS-3D and 3DPW could testify the performance of the proposed method in a more comprehensive way.

2. The way of showing qualitative videos is inconvenient for reader to watch. One video with more attractive demos / comparisons would help a lot.
Online multi-person 3D pose estimation and tracking in video with recurrent cross-attention
@inproceedings{
anonymous2024comotion,
title={CoMotion: Concurrent Multi-person 3D Motion},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=qKu6KWPgxt},
note={under review}
}
We introduce an approach for tracking detailed 3D poses of multiple people from a single monocular camera stream. Our system maintains temporally coherent predictions in crowded scenes filled with difficult poses and occlusions. Rather than detect poses and associate them to current tracks, our model directly updates all tracked poses simultaneously given a new input image. We train on numerous single-image and video datasets with both 2D and 3D annotations to produce a model that matches the 3D pose estimation quality of state-of-the-art systems while performing faster and more accurate tracking on in-the-wild videos.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
human pose estimation 3d human pose tracking
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/005394a658346a95d89b98ad07640a82833eea5a.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/25f9436054a9ca7b8f737b04029fd53f34a2e56d.zip
CoMotion: Concurrent Multi-person 3D Motion
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission