X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention
Portrait Animation;Head Avatar;Conditional Video Generation
Yes
No ethics review needed.
1.For non-human data, existing methods find it challenging to crop the face. If you were to apply these methods to such data, what kind of solution would you propose?
2.Could you simply train a model with added OOD data to provide results for non-human cases?
1.This paper is well written, easy to follow.
2.This paper proposes a new portrait animation pipeline that effectively addresses the longstanding issues of identity entanglement and loss of motion expressiveness.
3.Extensive experiments demonstrate the effectiveness of this method.
4.Great work! The motivation and experimental results for each component are solid. The demo in the supplementary materials also looks very impressive; (if it isn’t cherry-picked)
This paper proposes a novel portrait animation framework that extracts identity-free motion through a specially designed module and injects the motion using cross-attention, while utilizing GAN to enhance the accuracy of motion capturing. Extensive experiments demonstrate the effectiveness of this approach.
1.Since the motion model is trained, could it struggle to adapt to out-of-distribution (OOD) motions, could you provide extreme or unusual facial expressions to demonstrate robustness?
2.In the results provided in the paper and the demo, the facial features of the driving and reference are quite similar. Could you provide more examples where facial features (such as eyes, mouth, nose, etc.) or face position or head pose are inconsistent?
3.As stated in W2, I also cannot tell if this paper truly addresses the issue of identity leakage, as it appears that most of the features in the driving and reference images are quite similar, could you provide more convincing prof or experiments?
Yes
No ethics review needed.
1. In the quantitative comparisons, do the results for AniPortrait and X-Portrait come from the officially released weights or from weights that were retrained on your training dataset? If the latter, why do the results for both methods perform poorly in cases involving tongue motion, considering that the NerSemble dataset should include training samples with tongue motion? This is particularly concerning for X-Portrait, which is also a non-landmark method.
1. This work proposes a feasible solution to address the limitations of previous portrait animation methods that rely on explicit motion descriptors or the integration of motion information through PoseGuider and ControlNet.
2. This study demonstrates strong visual performance across various samples, showcasing its robust capabilities in motion transfer and stability.
3. This work includes comprehensive comparisons with prior methods and an ablation study to validate the proposed techniques.
This work presents a portrait animation method that diverges from traditional landmark-based approaches. It leverages a latent motion descriptor enhanced by a low-pass filter and incorporates motion priors through cross-attention, eliminating the reliance on aligned pose information. To develop a robust and fine-grained motion descriptor, the method includes a GAN head and employs techniques such as data augmentation and masked modeling.
1. A temporal evaluation of spatially aligned motion injection versus attention-based motion injection is recommended. Intuitively, spatially aligned motion injection is expected to provide better temporal consistency due to its stronger spatial priors.
2. Additional analysis and experiments are needed to clarify why X-NeMo achieves such high levels of temporal consistency. Other methods, such as LivePortrait and X-Portrait, also include a stage for training temporal modules, yet they still exhibit some flickering in their results. The paper mentions only temporal modules and prompt traveling as means to achieve temporal consistency, but I remain unclear about the source of X-NeMo's superior performance in this regard.

If these concerns are adequately addressed, I will consider raising my score to "accept."
Yes
The portrait animation has potential ethical impact. The authors briefly discuss it in P10.
Yes, Discrimination / bias / fairness concerns Yes, Privacy, security and safety
Please see the weakness.
+ The paper is well-structured.
+ The problem of portrait animation with high expressiveness and identity preservation is important.
+ The use of a 1D latent motion descriptor and cross-attention for motion control is reasonable.
The paper proposes X-NeMo, a diffusion-based portrait animation framework. It tries to address identity leakage and capturing diverse expressions. The method involves extracting a 1D identity-agnostic latent motion descriptor from the driving image, using cross-attention for motion control in image generation. It is trained end-to-end with a dual GAN decoder and spatial/color augmentations.
- What is the definition of zero-shot here? Firstly the model is trained. Secondly in the inference several reference images are provided. Thirdly the description of zero-shot is missing.
- The method has three training stages. What does it mean by end-to-end learning as described in several places in the paper? It seems each components are trained separately.
- The approach to get identity-agnostic feature is only to augment the images with color jitter, scaling and affine transformation. Such augmentations do not remove the identity information but only change the basic appearance of the facial images. The reviewer also believes such simple augmentations are not innovative.
- The app encode in Fig 2 seems to be not described.
- While the method shows plausible performance on the tested datasets, the generated video seem not to be tested with widely-used metrics like FVD.
Yes
No ethics review needed.
None
1.	The design of the implicit 1D latent motion descriptor and its integration through cross-attention offers a perspective on addressing identity leakage and expressiveness, overcoming the shortcomings of explicit motion descriptors.
2.	X-NeMO better captures subtle and extreme expressions in the process of portrait animation compared to previous models.
The paper introduces X-NeMo, a novel zero-shot, diffusion-based portrait animation framework that animates static portraits using head movements and facial expressions from a driving video of another subject. The authors identify key challenges in existing approaches, such as identity leakage and difficulty in capturing subtle and extreme facial expressions. To overcome these issues, X-NeMo employs a fully end-to-end training process that extracts a 1D identity-agnostic motion descriptor from the driving image, controlling the motion in the generated animation through cross-attention rather than traditional spatial guidance. This technique mitigates the transmission of identity clues from the driving video, reducing identity leakage and improving expressiveness. X-NeMo learns facial motion from diverse video datasets without relying on pre-trained motion detectors, using a dual GAN-based decoder and various augmentations to disentangle motion from identity cues. By embedding motion in a 1D latent vector and leveraging cross-attention, the model avoids transferring spatial structures, thereby preserving identity resemblance in the animated portraits. Experiments demonstrate that X-NeMo surpasses state-of-the-art methods, producing highly expressive animations while maintaining the subject’s identity.
1.	X-NeMo addresses the portrait animation task in the image level, and derive latent motion codes from each driving frame without the perception of the whole driving video. This setting neglect the video continuity, and may not capture the subtle coherence of expressions in a driving video.
@inproceedings{
anonymous2024xnemo,
title={X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=ML8FH4s5Ts},
note={under review}
}
We propose X-NeMo, a novel zero-shot diffusion-based portrait animation pipeline that animates a static portrait using facial movements from a driving video of a different individual. Our work first identifies the root causes of the limitations in prior approaches, such as identity leakage and difficulty in capturing subtle and extreme expressions. To address these challenges, we introduce a fully end-to-end training framework that distills a 1D identity-agnostic latent motion descriptor from driving image, effectively controlling motion through cross-attention during image generation. Our implicit motion descriptor captures expressive facial motion in fine detail, learned end-to-end from a diverse video dataset without reliance on any pre-trained motion detectors.  We further disentangle motion latents from identity cues with enhanced expressiveness by supervising their learning with a dual GAN decoder, alongside spatial and color augmentations. By embedding the driving motion into a 1D latent vector and controlling motion via cross-attention instead of additive spatial guidance, our design effectively eliminates the transmission of spatial-aligned structural clues from the driving condition to the diffusion backbone, substantially mitigating identity leakage. Extensive experiments demonstrate that X-NeMo surpasses state-of-the-art baselines, producing highly expressive animations with superior identity resemblance. Our code and models will be available for research.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Portrait Animation Head Avatar Conditional Video Generation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/a6781c1c886f951f0b4e3e67e76867d88fa254ed.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/7da3ead19be1faa4a6ec6f851376f8c5ac86480e.zip
X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission