TimeWalker: Personalized Neural Space for Life-long Head Avatar
Life-long Personalized Representation;Neural Parametric Morphable Model;
Yes
*Research integrity concerns
- The proposed approach seems to be an extended version of that of Pan et al. 2024. The technical contributions (Dynamo and DNA- 2DGS) are the same, the abstract is a verbatim copy, and there are two new loss functions: Lpips and consistency loss. Also, the proposed dataset adds 20 more actor collections than Pan et al. 2024. The experimental section shows nearly the same results (this paper submission just adds more results, and rendered faces seem to be in a different color space). Ironically, the ablation study of both papers (see Table 2  and 1 in the current submission and Pan et al. 2024, respectively) report the same results despite the slight differences in the final loss function. Why was Pan et al. 2024 not cited or attached to this submission? This is a major red flag on research ethics/integrity.

*Societal negative impact/potentially harmful applications
- Potential societal negative impacts around ethically sourced data (data origins, licensing, and consent) and misuse (strategies and guidelines for mitigating and detecting DeepFakes) must be addressed further in this paper (see Questions section for specific concerns).
Yes, Other reasons (please specify below)
*Section 3
- The introduction of the method in Section 3 (first two paragraphs) is redundant as most of it is already mentioned in the introduction. Please remove the first two paragraphs to release some space.
- Please move Section 3.1 (preliminaries) to the supplementary material, as more and more researchers are familiar with the basics of Gaussian Splatting.
The two suggestions above will give extra space to add more comparisons in the main papers.

*Comparisons
- Please move the comparisons with SoTA and mesh comparisons in the main paper.

*References and related work
- Please add the following references:
1) Caliskan et al. 2024. PAV: Personalized Head Avatar from Unstructured Video Collection. ECCV 2024
2) Pan et al. 2024. TimeWalker: Personalized Neural Space for Lifelong Head Avatars. CVPR 2024 Workshop POETS
The former paper is a dynamic NeRF-based approach that models an actor’s appearance and shape changes from an unstructured video collection. It utilizes a shared canonical shape space to encode identity and a deformed shape and appearance space to capture distinct colors and geometry from different videos captured at various times. Shape and appearance deformation are encoded through a multi-resolution hash grid and dynamic neural textures, respectively. The approach can naturally be adapted to model lifelong head avatars.
The latter paper is nearly identical to this paper submission but differs in some additional loss functions, dataset size, and number of applications. Why was this paper not cited or attached to this submission? This raises significant research integrity concerns to ICLR (see Details of Ethics Concerns).
- Please highlight the differences for 1) and especially 2), and provide quantitative and qualitative comparisons, especially for 2).

*Societal negative impact
- Please include an extended section on ethical considerations about the dataset and target applications. This could cover ethically sourced data concerns (data origins, licensing, and consent) as well as potential misuse scenarios, e.g., deepfakes. For the latter, please discuss additional mitigation strategies or guidelines for responsible use of the technology and detection of DeepFakes.
- Extended and more comprehensive version of that of Pan et al. 2024.
- Paper presents an extended version of TimeWalker-1.0 dataset introduced in Pan et al. 2024. It adds 20 more actor/celebrity image collections to the dataset. So, it should be called TimeWalker-2.0.
- The paper showcases a prompt-based 3D human head editing, demonstrating the Gaussian-based approach's generalization capabilities and flexibility.
The paper presents TimeWalker, a neural parametric representation that learns personalized representations of subjects with disentangled shape, expressions, and dynamic appearances across time.

The approach is an extension of that of Pan et al. 2024, which already proposes Dynamo and DNA-2DGS to efficiently model identity shape variations of the subject and expression-dependent dynamic texture and shape, respectively.

The main novelty of this paper boils down to two additional loss functions, 20 additional subjects added to the database, and a 3D face editing application that merges 2DGS and DGE.

As it stands, this is a system paper that shows minimal additional improvements (dataset and method) and one more application. The above limits the impact and contributions to the field.
- The proposed approach is nearly identical to that of Pan et al. 2024, yet the paper does not provide citations, discussions, or comparisons (see Questions section for specific concerns).
- The novelty and contributions are marginal: The proposed approach only adds additional loss functions, a larger dataset, and more applications to that of Pan et al. 2024.
- The proposed method does not advance the state of the art compared to previous methods (Caliskan et al. 2024 and Pant et al. 2024).
- The proposed dataset might raise ethical concerns as celebrities and actors have not consented to using their images and videos to generate and manipulate sensitive content, such as their faces. Discussions around ethically sourced data, e.g., the origin of the data, licensing, and consent, should be addressed and discussed.
- The paper must further address the potential societal negative impact, especially around Deepfakes. The authors should discuss additional ways to mitigate misuse (see Questions section for more details).
Yes
This paper introduces a new dataset including 40 celebrities.
Yes, Responsible research practice (e.g., human subjects, data release)
- How fast is the rendering of this method during inference?
- The method does not evaluate the identity preservation of the person when controlling for lifestage and shape.
- When switching between different life stages, do you consider the order of switching time? Or multiple life stages switched randomly?
- It seems that the Gaussian and 3DMM of this method are bound together (during initialization). Why is it that when it is driven by moment (change the expression), it does not directly use 3DMM to deform the Gaussian points, but introduces a warp field instead?
- In Table 5 (in 1 vs N), FlashAvatar reports lower results than INSTA. The experiments (1vsN) has a similar setting to FlashAvatar paper, but the results is inconsistent with the statement in the FlashAvatar paper that FlashAvatar performs better than INSTA. What could be the possible reason?
- Please refer to the weakness.
- The article can learn head reconstruction from a set of videos and pictures of the target object at different times, and change between the image and shape at different times after learning is completed.
- This paper includes a dataset of 40 celebrities.
This paper proposes TimeWalker: a scheme for modeling lifelong head portraits. The method learns from a sequence of images and videos of the target object.
- The visual effects of this method have many artifacts, which can usually be avoided when using a large amount of data (15k-260k per person). In addition, the transitions between different time periods are not natural enough, which has a significant impact on achieving the main goal of this method: seamlessly altering their appearance within a predefined set of life stages. For example, in the Personalized Space provided by the project website, the texture of the skin and the overall blurriness are unsatisfactory, and the lifestage switching looks like a quick and abrupt switch between videos of several life periods. The authors should also consider showing more seamless switching results.
- Most importantly, the approach has not been thoroughly compared, making it difficult to assess its contribution. For example, the method lacks discussion of recent Gaussian head methods[1,2], but compared to the baseline methods (insta) of these papers[1,2],
- There is also a lack of comparison with one-shot head reconstruction methods [3,4,5,6], which can also easily reconstruct the target person's head of various ages and shapes, given that in our setting, we have data of the target person at different ages. Although these one-shot methods cannot achieve seamless switching between different shapes and life cycles, this method is also not fully evaluated in this regard (see questions item 2,3).
- While there is a similar paper (https://openreview.net/forum?id=3cUdmVfRb4) published in non-archived form and allowed to be resubmitted in archived form, the current submission reuses a lot of text/visualizations/quantitative results, importantly the methodological and data details are different while the experimental results are the same. For example, the dataset TimeWalker-1.0 proposed in the non-archived paper includes 20 celebrities, but in this submission there is 40. But the ablation study table: table 2 in this submission reused all the numbers in table1 of the non-archived paper.

- Reference:

        [1] Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians
        [2] SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
        [3] Generalizable One-shot Neural Head Avatar
        [4] Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis
        [5] GPAvatar: Generalizable and Precise Head Avatar from Image(s)
        [6] Portrait4D-v2: Pseudo Multi-View Data Creates Better 4D Head Synthesizer
Yes
No ethics review needed.
1. Why not use a general basis instead of having a specific head basis for each person?
2. How many head bases are learned for each person?
3. What is the inference speed?
1.  They present a new avatar setting that focuses on capturing a comprehensive head avatar over a lifelong scale. To achieve this, they introduce a dataset where each ID contains 12K to 260K frames, spanning diverse age groups and head variations.
2.  This paper proposes a Dynamo module, which learns a set of head basis that could represent the comprehensive head variations of the target person over ages. 
3.  They introduce DNA-2DGS to model head motion deformations based both on head basis and FLAME parameters.
4.  Extensive experiments are conducted to verify the effectiveness of the proposed method.
This paper proposes to reconstruct a 3D head avatar of a person on a lifelong scale. Specifically, unlike previous avatars that are limited to moment-level reconstructions, this paper achieves full reconstruction and animation of a person across different ages. To this end, they introduce a new head parameter model, in which they design a set of head basis. By linearly combining these bases, they can model a person’s face at different life moments. Additionally, they propose a DNA-2DGS to better integrate the head basis with the deformation from the original FLAME model, resulting in realistic head deformation modeling. They also introduce a dataset where each ID consists of 12K -260K frames across diverse age and head distributions.
1.  The specific role of this residual embedding is unclear. Is it because the expressiveness of the head basis is insufficient? There seems to be a lack of ablation studies. 
2.  Some details are missing. For example, during inference, how are the blend weights for head basis determined? For instance, when the age is 20, how is the age-to-blend weight mapping obtained? Additionally, is the  I_{res}  that was consistent for all age stages?
3.  The results contain several artifacts, especially during eye-blinking and around the head edges.
4.  In the video results, different life stages exhibit varying lighting conditions. That means several age-irrelevant attributes, such as lighting, were not decoupled from age.

**Similar/identical paper published at CVPR 2024 Workshop POETS 2nd Round**
Thanks for Reviewer f3xJ's remainder.
After reading the paper https://openreview.net/forum?id=3cUdmVfRb4,  I found that this paper is very similar to it, even with identical experimental values (Ablation Study). Based on these findings, I decided to reject this paper.
Yes
No ethics review needed.
1. Regarding the FLAME parameters of the dataset:  
   1. What method was employed to obtain the FLAME parameters for the constructed dataset?  
   2. How accurate are these FLAME parameters? Additionally, how might the accuracy of the preprocessed FLAME parameters impact the fidelity of the final avatars?

2. Is the representation of lifetime spaces continuous?

3. What is the performance of the method from different viewpoints? The results presented primarily focus on frontal views.

4. The specific lifetime avatars exhibit noticeable shadowing. Given the videos captured at different times, could it be possible to mitigate these shadows to enhance the visual quality of the results?
1. The paper addresses an intriguing and technically significant task, and the constructed dataset will be valuable for future research in this domain.

2. The proposed method is technically robust, and the extensive experiments provide strong justification for its effectiveness.
The paper presents a novel framework to model lifelong 3D animatable avatars. To address this avatar reconstruction problem on a lifelong scale, the author first constructs a large-scale dataset comprising lifelong images from different subjects. Then, they propose a head representation using neural head basis and 2D Gaussian Splatting (GS) for modeling all life stages of one person. Extensive experiments demonstrate the proposed approach can effectively reconstruct lifelong 3D animatable avatars.
1. The reconstructed lifelong avatars exhibit significant artifacts beyond those associated with the mouth region, as noted by the authors. However, there is a lack of discussion regarding the origins of these artifacts. For instance, could the limited availability of data during certain life stages be a contributing factor?

2. It would be beneficial to explore the advantages of incorporating multiple lifetimes within the representation for this task. However, the paper lacks a discussion on this point, particularly concerning the extent of performance improvement attributed to the inclusion of additional lifelong data.

3. The novelty of the proposed method appears to be somewhat constrained. The use of blendshape concepts within the Gaussian Splatting framework bears similarities to prior work [1]. Additionally, the adoption of the multi-resolution Hashgrid has also been utilized in INSTA [2], further limiting the originality of the approach.


[1] Ma, Shengjie, et al. "3d gaussian blendshapes for head avatar animation." *ACM SIGGRAPH 2024 Conference Papers*. 2024.

[2] Zielonka, Wojciech, Timo Bolkart, and Justus Thies. "Instant volumetric head avatars." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
We present TimeWalker, a new paradigm that realistically models a complete 3D head avatar replica of a person on life-long scale.
@inproceedings{
anonymous2024timewalker,
title={TimeWalker: Personalized Neural Space for Life-long Head Avatar},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=gNWTngNX31},
note={under review}
}
We present TimeWalker, a novel framework that models realistic, full-scale 3D head avatars of a person on lifelong scale. Unlike current human head avatar pipelines that capture a person's identity only at the momentary level (i.e., instant photography, or short videos), TimeWalker constructs a person's comprehensive identity from unstructured data collection over his/her various life stages, offering a paradigm to achieve full reconstruction and animation of that person at different moments of life.  At the heart of TimeWalker's success is a novel neural parametric model that learns personalized representation with the disentanglement of shape, expression, and appearance across ages. Central to our methodology are the concepts of two aspects: (1) We track back to the principle of modeling a person's identity in an additive combination of his/her average head representation in the canonical space, and moment-specific head attribute representations driven from a set of neural head basis. To learn the set of head basis that could represent the comprehensive head variations of the target person in a compact manner, we propose a Dynamic Neural Basis-Blending Module (Dynamo). It dynamically adjusts the number and blend weights of neural head bases, according to both shared and specific traits of the target person over ages. (2) We introduce Dynamic 2D Gaussian Splatting (DNA-2DGS), an extension of Gaussian splatting representation, to model head motion deformations like facial expressions without losing the realism of rendering and reconstruction of full head. DNA-2DGS includes a set of controllable 2D oriented planar Gaussian disks that utilize the priors from a parametric morphable face model, and move/rotate with the change of expression.  Through extensive experimental evaluations, we show TimeWalker's ability to reconstruct and animate avatars across decoupled dimensions with realistic rendering effects, demonstrating a way to achieve personalized ``time traveling'' in a breeze.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Life-long Personalized Representation;Neural Parametric Morphable Model;
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/80d02343a677d4682655cb11c3a69894a1b449ad.pdf
generative models
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/074418a2cd8a1fc699963373c333e770618598c0.zip
TimeWalker: Personalized Neural Space for Life-long Head Avatar
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission