CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
RL;PPO;motion;motion generation;motion synthesis;synthesis;generative models;diffusion;animation
Yes
No ethics review needed.
Please see weaknesses above.
The paper is well-written, with the key contributions listed clearly in the introduction. The authors also identified relevant literature in this line of research and highlighted their contributions in the method. The authors also included low-level technical details which aid to the reproducibility of the work. Experimental evaluations highlight CLoSD’s superior performance on multi-task benchmarks, including goal-reaching, object striking, sitting and getting up. In tasks requiring environmental interaction, CLoSD significantly outperforms both state-of-the-art open-loop diffusion models and other text-to-motion RL systems, demonstrating its ability to maintain stable, responsive control across diverse physical interactions. Overall, I believe this paper is a good step towards understanding the interaction between generative models and physical simulation.
This paper presents CLoSD, a text-driven reinforcement learning (RL) controller for character motion, merging the generative capabilities of diffusion models with the physical realism of RL-based control in a closed-loop framework. The proposed framework has two components: a Diffuion Planner (DiP), and a RL tracking controller. DiP is an auto-regressive diffusion model that is controlled by textual prompts and target locations and outputs motion plans. The RL tracking controller, based on the Perpetual Humanoid Control (PHC) model, interprets DiP’s motion plans and executes them in a physics simulation.
Recent generative models are capable of generating plans in much larger observation spaces. Can the authors comment on how this work can involve sensory inputs (if at all), such as vision, to enhance CLoSD's adaptability in more complex environments? Moreover, can the authors comment on the potential failure modes of their method if their method was used for a longer planning horizon to generate even more intricate motion sequences?
Yes
No ethics review needed.
**Tracking Policy**

I have a few questions regarding the PHC setup. It is stated that a simplified version of PHC was used, excluding the mixture of experts. 
* Does this imply that only a single primitive policy was trained? If so, is the controller essentially a standard policy trained solely with AMP and tracking rewards? 
* Additionally, how would the full PHC version perform in comparison? 
* Would an alternative controller, such as the one proposed by Chentanez et al. 2018, which incorporates future frames, potentially be more effective?

There has been more recent work on character control for tracking reference motions, including newer controllers developed in 2024. 
* Consider referencing newer motion tracking controllers to provide a more current perspective. 
* Additionally, other methods have also explored controllers that operate without a reference motion. While these are not directly relevant, It would be beneficial to discuss such approaches, including whether the integration of a DiP-like approach is feasible in this context. 

**Comparisons with UniHSI**

UniHSI uses an LLM with scene information to design a chain of contacts. Their demonstrated results look less smooth than CLoSD; however, they look much better than those presented here. For example, their pipeline shows successful „stand-up “ performance. As far as I understand, in the comparisons here, the LLM Planner was omitted, and instead, a fixed target label for the pelvis alone is defined. 
* The discrepancy between UniHSI’s original results and the lower-quality outcomes here needs clearer explanation and precise clarification. 
* Further UniHSI shows more complex chains of tasks, sitting on different objects, lying down etc. Can CLoSD also sit on variable hights? where are the limits of versatility?

**Text-to-Motion**
* PhysDiff’s metrics are used to evaluate the physical correctness of the text-to-motion quality compared to MDM and MoConVQ. It seems more intuitive to use PhysDiff (MDM + Physical Projection) as an additional comparison to understand how much the closed-loop approach improves over the proposed physical projection in PhysDiff.
* As far as I understand, the CLoSD results are the motions performed in the physics-based simulator. This makes me wonder why there is still such a high floating error of 2cm on average.
* It would be helpful to visualize a side-by-side comparison of DiP and MDM motions, as DiP appears to significantly outperform MDM in this aspect.
* A comparison between DiP and CAMDM is needed to clarify why DiP might be better suited than the Chen et al. 2024 architecture mentioned in line 236.

**Motion Quality**
* The presented motions in the video are not very smooth and contain a lot of shakiness and artifacts that seem to trick the physics (e.g., fast vibrations allowing the character to drift - 02:42)
* In the renderings with the SMPL mesh, strong artifacts in hand pose and feet are visible. Can you explain what is causing those artifacts that are less severe in MDM.
* In the last motion in the video, “A person is waving goodbye,” the model attempts a simple motion that MDM typically handles with high precision, yet here it appears less accurate. This raises the question of whether CLoSD is overfitted to the specific tasks, potentially limiting its ability to leverage the full capabilities of DiP/MDM in terms of motion quality and diversity.

**Limitations**
* MDM can generate a diverse range of dynamic motions, and PHC can track many AMASS motions. However, the motions presented here appear more constrained. A more detailed explanation of the method’s limitations would be helpful.
* The method’s applications rely heavily on various tricks, making it challenging to determine how much of the outcome is due to human engineering versus the method’s inherent capabilities. For instance, to ensure the object is hit, the user must specify a joint target location and an appropriate prompt. Consequently, the approach appears overfitted to this specific set of motions/tasks and does not fully utilize the generative capabilities of the diffusion model. It would be helpful if the authors could discuss these limitations further.
* The goal of the paper is clear: combine the strength of a DIffusion Model (MDM) with a physics-based controller (PHC).
* The method is simple yet interesting enough.
* The paper is well-written, and the method is clearly explained.
* Code will be publicly available.
This work presents a method that combines a diffusion-based motion planner with a tracking controller in a closed-loop setup. The motion planner is an autoregressive diffusion model that predicts future character poses based on previous states and conditioning inputs, such as joint target positions and text descriptions. The controller is an off-the-shelf state-of-the-art tracking controller finetuned to the motions generated by the DiP. The result is a text-to-motion controller that can be used to solve different tasks while having control over the style of the motions.
* While the method is well explained, the experimental section lacks clarity and precision. (see *Questions*)
* The experiments could be better designed. 
* The video shows lower motion quality than state-of-the-art physics-based character control, which the paper does not address.
* The limitations are not clearly stated and discussed.
Yes
No ethics review needed.
- I am curious about the difference between open-loop and closed-loop approaches. In Table 1, the open-loop setting performs well on “goal reaching” and “object striking” tasks but struggles with “sitting” and “getting up” tasks. Could you elaborate on why re-planning is necessary in these cases?
- $x^{\text {prefix }}$ is defined as “current frames,” but does this risk the model forgetting its previous plans? In other words, might it be more effective to define $x^{\text {prefix }}$ as motion tracking errors, or are there alternative design choices that could work better?
- This paper combines a diffusion planner with an RL motion tracker to achieve natural and versatile physics-based character control across multiple tasks.
- CloSD outperforms previous methods, such as InterPhys and UniHSI, and provide a solid framework for future research.
- Extensive results and ablation studies demonstrate the effectiveness of each design choices and influence of hyper-parameters.
- The paper is overall well-structured and well-written.
This paper presents CloSD, a method that combines a diffusion planner with a reinforcement learning tracking policy to enable multi-task character control. The results demonstrate that CloSD achieves natural and versatile motions.
Please refer to the questions part.
Yes
No ethics review needed.
See weaknesses
1. The paper introduces a closed-loop integration between motion diffusion and RL, leveraging each method’s strengths and overcoming traditional limitations, such as the need for pre-planned motion sequences or manual fine-tuning.
2. CLoSD’s capability to execute multiple tasks through simple text prompts and target positions highlights its flexibility and generalization, reducing the need for task-specific controllers.
3. By optimizing the Diffusion Planner to be both autoregressive and fast, CLoSD maintains responsiveness in dynamic environments, a crucial factor for real-world applications.
The paper presents CLoSD, a multi-task character motion generation approach that combines text-driven motion geneation models and reinforcement learning to control physics-based simulations of human-like motion. By combining motion diffusion, which generates a variety of text-based motions, and RL, which enables interaction with the environment, CLoSD ensures both diversity and physical accuracy in character motion. The system operates in a closed loop, where a Diffusion Planner (DiP) generates motion plans from text prompts and target locations, and a tracking controller uses these plans to adapt the character’s actions in real-time. This setup enables motion generation across tasks like navigating to a target, interacting with objects, and completing actions like sitting or getting up. The approach demonstrates high task success rates, adaptability to on-the-fly text changes, and a significant improvement over existing controllers, as it allows characters to perform varied tasks while adhering to realistic physical constraints.
1. The paper could benefit from better contextualization. While the integration of CAMDM and PHC for multitask character control is intriguing and effectively combines a diffusion model for language-conditioned motion generation with a tracker to ensure physical feasibility, the Related Work section would benefit from a deeper discussion since similar architecture has been explored in previous works like Trace and Pace or InsActor. Specifically, while it briefly mentions these methods, it lacks depth on the specific contributions of CLoSD that differ from such hierarchical decomposition in language-conditioned motion generation as explored in prior works. Including insights on how CLoSD advances or contrasts with these methods, such as motion in-painting, goal-reaching, and motion chaining demonstrated in InsActor, could enhance readers’ understanding of the unique contributions of this work.
2. The novelty of the work could be better conveyed. While the combination of CAMDM and PHC for multitask character control aligns with existing architectures, CLoSD’s main distinguishing feature is its real-time generation capability, which largely stems from the use of CAMDM. A clearer exposition of the unique contributions and advancements CLoSD introduces—beyond real-time generation—would help emphasize its originality and innovation.
3. The experimental evaluation is somewhat limited in scope. Although the paper includes experiments on motion generation with HumanML3D, a broader range of tasks would provide a more comprehensive validation of the approach. Since the method is intended to enable character interactions with environmental objects, including more diverse interaction tasks beyond the four basic ones presented would greatly enrich the empirical support and demonstrate the approach’s robustness and generalizability.

Overall, the paper presents an interesting approach to interactive character control. However, I would appreciate a deeper discussion that thoroughly compares the proposed work with prior works.
@inproceedings{
anonymous2024closd,
title={{CL}o{SD}: Closing the Loop between Simulation and Diffusion for multi-task character control},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=pZISppZSTv},
note={under review}
}
Motion diffusion models and Reinforcement Learning (RL) based control for physics-based simulations have complementary strengths for human motion generation. The former is capable of generating a wide variety of motions, adhering to intuitive control such as text, while the latter offers physically plausible motion and direct interaction with the environment. In this work, we present a method that combines their respective strengths. CLoSD is a text-driven RL physics-based controller, guided by diffusion generation for various tasks. Our key insight is that motion diffusion can serve as an on-the-fly universal planner for a robust RL controller. To this end, CLoSD maintains a closed-loop interaction between two modules — a Diffusion Planner (DiP), and a tracking controller. DiP is a fast-responding autoregressive diffusion model, controlled by textual prompts and target locations, and the controller is a simple and robust motion imitator that continuously receives motion plans from DiP and provides feedback from the environment. CLoSD is capable of seamlessly performing a sequence of different tasks, including navigation to a goal location, striking an object with a hand or foot as specified in a text prompt, sitting down, and getting up.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
RL PPO motion motion generation motion synthesis synthesis generative models diffusion animation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/985c780b2428e728b6f8ea075f9fab9c0548db77.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/ac0c39fe9a9a19efcea4db561fb95611412da8cb.zip
CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission