International Journal of Computer Vision manuscript No.
(will be inserted by the editor)

Learning from Longitudinal Face Demonstration Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

arXiv:1711.10520v2 [cs.CV] 20 Feb 2018

Chi Nhan Duong · Kha Gia Quach · Khoa Luu · T. Hoang Ngan Le · Marios
Savvides · Tien D. Bui

Received: date / Accepted: date

Abstract This paper presents a novel Subject-dependent Deep
Aging Path (SDAP), which inherits the merits of both Generative Probabilistic Modeling and Inverse Reinforcement
Learning to model the facial structures and the longitudinal
face aging process of a given subject. The proposed SDAP
is optimized using tractable log-likelihood objective functions with Convolutional Neural Networks (CNNs) based
deep feature extraction. Instead of applying a fixed aging development path for all input faces and subjects, SDAP is able
to provide the most appropriate aging development path for
individual subject that optimizes the reward aging formulation. Unlike previous methods that can take only one image
as the input, SDAP further allows multiple images as inputs,
i.e. all information of a subject at either the same or different
ages, to produce the optimal aging path for the given subject. Finally, SDAP allows efficiently synthesizing in-thewild aging faces. The proposed model is experimented in
both tasks of face aging synthesis and cross-age face verification. The experimental results consistently show SDAP
achieves the state-of-the-art performance on numerous face
aging databases, i.e. FG-NET, MORPH, AginG Faces in the
Wild (AGFW), and Cross-Age Celebrity Dataset (CACD).
Furthermore, we also evaluate the performance of SDAP on
large-scale Megaface challenge to demonstrate the advantages of the proposed solution.

Fig. 1: An illustration of age progression using direct
(IAAP), step-by-step (TNVP) and our SDAP approaches.
Instead of using a fixed aging development path for all inputs as in IAAP (the 1st row) and TNVP (the 2nd row),
given a face and the target age, SDAP can find the optimal
aging development path (the 3rd row) for the subject. SDAP
can also collect aging cues from multiple inputs to produce
more aesthetical synthesis results (the 4th row).

Keywords Face Age Progression; Deep Generative
Models; Face Aging; Subject-dependent Deep Aging;
Tractable Graphical Probabilistic Models.

1 Introduction
Chi Nhan Duong1 , Kha Gia Quach1 , Khoa Luu1 , T. Hoang Ngan
Le1 , Marios Savvides1 , Tien D. Bui2
1
CyLab Biometrics Center and the Department of Electrical and
Computer Engineering, Carnegie Mellon University, Pittsburgh, PA,
USA.
2
Department of Computer Science and Software Engineering,
Concordia University, Montréal, Québec, Canada.
E-mail: {chinhand, kquach, kluu, thihoanl}@andrew.cmu.edu,
msavvid@ri.cmu.edu, bui@encs.concordia.ca

The problem of face aging targets on the capabilities to aesthetically synthesize the faces of a subject at older ages,
i.e. age progression, or younger ages, i.e. age regression or
deaging. This problem is applicable in various real-world
applications from age invariant face verification, finding missing children to cosmetic studies. Indeed, face aging has raised
considerable attentions in computer vision and machine learning communities recently. Several breakthroughs with numerous face aging approaches, varying from anthropology

2

Chi Nhan Duong et al.

Table 1: The comparison of the properties between our SDAP approach and other age progression methods. Deep learning (DL), Inverse Reinforcement Learning (IRL), Dictionary (DICT), Prototype (PROTO), Probabilistic Graphical Models
(PGM), Log-likelihood (LL), Adversarial (ADV). Note that 7 indicates unknown or not directly applicable properties.

Subject-dependent Aging Path
Multiple Input Optimization
Tractable
Non-Linearity
Model Type
Architecture
Loss Function

Our SDAP

TNVP
(Duong
et al, 2017)

CAAE
(Zhang
et al, 2017)

3
3
3
3
DL + IRL
PGM+CNN
LL

7
7
3
3
DL
PGM+CNN
LL

7
7
3
3
DL
CNN
ADV+`2

theories to deep learning structures have been presented in
literature. However, the synthesized face aging results in
these previous approaches are still far from perfect due to
various challenging factors, such as heredity, living styles,
etc. In addition, face aging databases used in most methods
to learn the aging processes are usually limited in both number of images per subject and the covered age ranges of each
subject.

RNN
(Wang
et
al,
2016)
7
7
3
3
DL
CNN
`2

TRBM
(Duong
et
al,
2016)
7
7
7
3
DL
PGM
LL

HFA
(Yang
et
al,
2016)
7
7
3
7
DICT
Bases
LL+`0

IAAP
(KemelmacherShlizerman
et al, 2014)
7
7
7
7
PROTO
7
7

ate progressed faces from one age group to the next until reaching the target. These modeling structures can efficiently learn the temporal information and provide more
age variation even when a target age is very far from the input age of a subject. However, the main limitation of these
methods is the lack of longitudinal face aging databases. The
longest training sequence usually contains only three or four
images per subject.

Both conventional and deep learning methods usually inLimitations of previous approaches. In either directions (i.e.
clude two directions, i.e. direct and step-by-step aging syndirect or step-by-step aging synthesis) the aging approach
thesis, in exploring the temporal face aging features from
falls in, these previous approaches still suffer from many
training databases. In the former direction, these methods
challenging factors and remain with lots of limitations. Tadirectly synthesize a face to the target age using the relationble 1 compares the properties of different aging approaches.
ships between training images and their corresponding age
labels. For example, the prototyping approaches (Burt and
– Non-linearity. Since human aging is a complicated and
Perrett, 1995; Kemelmacher-Shlizerman et al, 2014; Rowhighly nonlinear process, the linear models mostly used
land et al, 1995) use age labels to organize images into age
in conventional methods (i.e. prototype, AAMs-based
groups and compute average faces for their prototypes. Then,
and 3DMM-based approaches) are not able to efficiently
the difference between source-age and target-age prototypes
interpret the aging variations and the quality of their synis applied directly to the input image to obtain the age-progressed thesized results is very limited.
face at the target age. Similarly, the Generative Adversarial
– Loss function of deep structure. The use of a fixed reNetworks (GAN) approach (Zhang et al, 2017) models the
construction loss function, i.e. `2 -norm, in the proposed
relationship between high-level representation of input faces
deep structures, i.e. (Wang et al, 2016; Yang et al, 2016),
and age labels by constructing a deep-neural-network genusually produces blurry synthesis results.
erator. This generator is then incorporated with the target
– Tractability. Exploiting the advantages of probabilistic
age labels to synthesize the outputs. Although these kinds of
graphical models has introduced a potential direction for
models are easy to train, they are limited in capabilities to
deep model design and produced prominent synthesized
synthesize faces much older than the input faces of the same
results for the age progression task (Duong et al, 2016).
subject, e.g. directly from ten to 60 years old. Indeed, the
– Data usability. Even though a subject in training/testing
progression of a face at ten years old to the one at 60 years
set has multiple images at the same age, there is only one
old in these methods usually ends up with a synthesized face
image used to learn/synthesize in these methods. The
using 10-year-old features plus wrinkles.
other images are usually wastefully ignored. In addition,
Meanwhile, the latter approaches (Duong et al, 2017,
2016; Shu et al, 2015; Wang et al, 2016; Yang et al, 2016)
decompose the long-term aging process into short-term developments and focus on the aging transform embedding between faces of two consecutive development stages. Using
learned transformation, these methods step-by-step gener-

the aging transformation embedding in these approaches
is only able to proceed on images from two age groups.
– Fixed aging development path. The learned aging development path is identically applied for all subjects which
is not true in reality. Instead, each subject should have
his/her own aging development.

Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

3

Contributions of this work. The paper presents a novel Subject- AGES using guidance faces corresponding to the subject’s
characteristics to produce more stable results. Texture syndependent Deep Aging Path (SDAP) model to face age prothesis was also combined in the later stage to produce better
gression, which is an extension of our previous work (Duong
facial details. (Suo et al, 2010, 2012) introduced the threeet al, 2017). In that work, TNVP structure is proposed to emlayer And-Or Graph (AOG) of smaller parts, i.e. eyes, nose,
bed the pairwise transformations between two consecutive
mouth, etc., to model a face. Then, the face aging process
age groups. In this work, the SDAP structure is introduced
was learned for each part using a Markov chain.
to further enhance the capability to discover the optimal agReconstruction-based aging methods model aging faces
ing development path for each individual. This goal can be
by unifying the aging basis in each group. (Yang et al, 2016)
done by embedding the transformation over the whole aging
represented person-specific and age-specific factors indepensequence of a subject under an IRL framework. Our contridently using sparse representation hidden factor analysis (HFA).
butions can be summarized as follows.
(Shu et al, 2015) presented the aging coupled dictionaries
1. The aging transformation embedding is designed using
(CDL) to model personalized aging patterns by preserving
(1) a tractable log-likelihood density estimation with (2)
personalized facial features.
Convolution Neural Network (CNN) structures and (3)
Prototyping-based aging methods employ the age proa present of an age controller to indicate the amount of
totypes to produce new face images. The average faces of
aging changes for synthesis. Thus, the proposed SDAP
all age groups are used as the prototypes (Rowland et al,
is able to provide a smoother synthesis across faces be
1995). Then, input face image can be progressed to the target
produced and give the usability of aging data, i.e. all image by incorporating the differences between the prototypes
ages of a subject in different or the same ages, is also
of two age groups (Burt and Perrett, 1995). (Kemelmacherimproved for better transformation.
Shlizerman et al, 2014) presented to construct high quality
2. Unlike most previous methods, our proposed SDAP model average prototypes from a large-scale set of images. The
further enhances the capability to find the optimal aging
subspace alignment and illumination normalization were also
development path for individual. This goal can be done
included in this system.
by embedding the transformation over the whole aging
Deep learning-based aging approaches have recently achieved
sequence of a subject under an IRL framework.
considerable results in face age progression using the power
3. Instead of using pre-defined or add-hoc aging reward
of deep learning. (Duong et al, 2016) introduced Tempoand objective functions as in most previous work, our
ral Restricted Boltzmann Machines (TRBM) to represent
proposed approach allows the algorithm to automatically
the non-linear aging process with geometry constraints and
come up with the optimal objective formulation and paspatial RBMs to model a sequence of reference faces and
rameters via a data driven strategy in training.
wrinkles of adult faces. (Wang et al, 2016) approximated
aging sequences using a Recurrent Neural Networks (RNN)
We believe that this is the first work that designs an IRL
with two-layer Gated Recurrent Unit (GRU). Recently, the
framework to model the longitudinal face aging.
structure of Conditional Adversarial Autoencoder (CAAE)
is also applied to synthesize aged images in (Antipov et al,
2017). (Duong et al, 2017) proposed a novel generative prob2 Related work
abilistic model, called Temporal Non-Volume Preserving (TNVP)
transformation, to model a long-term facial aging process as
This section reviews recent methods in face age progression.
a sequence of short-term stages.
These methods can be technically classified into four categories, i.e. modeling, reconstruction, prototyping, and deep
learning-based methods.
3 Our Proposed SDAP
Modeling-based aging is one of the earliest categories
presented face age progression. These methods usually model TNVP structure has provided an efficient model to capture
the pairwise transformation between faces of consecutive
both facial shapes and textures using a set of parameters, and
age groups (Duong et al, 2017). However, it still has some
learn the face aging process via aging functions. (Patterson
limitations. Firstly, the TNVP mainly focuses on the pairet al, 2006) and (Lanitis et al, 2002) employed a set of Acwise relationship rather than the long-term relationship pretive Appearance Models (AAMs) parameters with four agsented in an aging sequence. Secondly, capability of aping functions to model both the general and the specific agplying different development paths for different subjects is
ing processes. Luu et al. (Luu et al, 2009) combined familstill absent. In reality, each subject should have his/her own
ial facial cues to the process of face age progression. (Geng
et al, 2007) presented AGing pattErn Subspace (AGES) method aging development progress because each person ages differently. In this section, we introduce a more flexible structo construct a subspace for aging patterns as a chronological
ture, named Subject-dependent Deep Aging Path (SDAP),
sequence of face images. (Tsai et al, 2014) then enhanced

4

Chi Nhan Duong et al.

with an additional component, i.e. an age controller. This
age controller provides the capability of defining how much
age variation should be added during synthesis. This architecture, therefore, benefits both training stages, i.e. by maximizing the usability of training aging data, and testing stage,
i.e. becoming more flexible to adopt different aging path
to different subjects according their features. Moreover, instead of only learning from image pairs of a subject in two
consecutive age groups, SDAP has the capability of embedding the aging transformation from longer aging sequences
of that subject which efficiently reflects the long-term aging development of the subject. We also show that goal can
be achieved under an Inverse Reinforcement Learning (IRL)
framework. The structure of this section is as follows: We
first present our novel approach to model the facial structures in Subsection 3.1. Then, our IRL learning approach to
the longitudinal face aging modeling is detailed in Subsection 3.2.

Fig. 2: The structures of a Synthesis Component as a composition of Synthesis Units and an example of a mapping
function with two mapping units.
The interactions between latent variables {zt−1 , zt } and
the aging controller variable at−1 are 3-way multiplicative.
They can be mathematically encoded as in Eqn. (2).
git =

The proposed architecture consists of three main components, i.e. (1) latent space mapping, (2) aging transformation, and (3) age controller. Our age controller provides the
capability of defining how much age variation should be
added during synthesis. Using this structure, our model is
flexible to aging in different ways corresponding to the input faces. Moreover, it also helps to maximize the usability
of training aging data.
Structures and Variable Relationship Modeling: Our graphical model (Fig. 2) consists of three sets of variables: observed variables {xt−1 , xt } ∈ I encoding the textures of
face images in the image domain I ⊂ RD at two stages t−1
and t; their corresponding latent variables {zt−1 , zt } ∈ Z in
the latent space Z; and an aging controller variable at−1 ∈
A ⊂ RNa . The aging controller at−1 is represented as a
one-hot vector indicating how many years old the progression process should perform on xt−1 . The bijection functions F1 , F2 , mapping from the observation space to the latent space, and the aging transformation G are defined as in
Eqn. (1).
→Z

xt−1

7→ zt−1 = F1 (xt−1 ; θ 1 )

xt

7→ z̄t = F2 (xt ; θ 2 )

G :Z, A

ŵijk zjt−1 at−1
+ bi
k

(2)

j,k

3.1 Aging Embedding with Age Controller

F1 , F2 :I

X

where Ŵ ∈ RD×D×Na is a 3-way tensor weight matrix
and b is the bias of these connections. Eqn. (2) enables two
important properties in the architecture. First, since at−1 is
an one-hot vector, different controllers will enable different
sets of weights to be used. Thus, it allows controlling the
amount of aging information to be embedded to the aging
process. Second, given the age controller, the model is able
to use all images of a subject to enhance its performance.
In practice, the large number of parameters of the 3-way
tensor matrix may have negative effects to the scalability
of the model. Thus, Ŵ can be further factorized into three
matrices Wa ∈ Rf ×Na , Wz ∈ Rf ×D , and W ∈ RD×f
with f factors by adopting (Taylor and Hinton, 2009) as in
Eqn. (3).
gt = W(Wz zt−1

where

Wa at−1 ) + b

(3)

stands for the Hadamard product.

The Log-likelihood: Given a face xt−1 in the age group t −
1, the probability density function can be formulated as,
pX t (xt |xt−1 , at−1 ; θ) = pX t (xt |zt−1 , at−1 ; θ)
∂zt
(4)
∂xt
t
t−1
t−1
t
pZ t ,Z t−1 (z , z
|a
, θ) ∂z
=
pZ t−1 (zt−1 ; θ)
∂xt

= pZ t (zt |zt−1 , at−1 ; θ)
(1)

→Z

zt−1 , at−1 7→ gt = G(zt−1 , at−1 ; θ 3 )

where θ = {θ 1 , θ 2 , θ 3 } denotes the set of parameters of
F1 , F2 , G, respectively. Notice that in SDAP, the structure
of bijection functions is adopted from TNVP architecture.
Then, the relationship between latent variables zt , zt−1 and
at−1 is computed as zt = gt + z̄t .

where pX t (xt |xt−1 , at−1 ; θ) and pZ t (zt |zt−1 , at−1 ; θ) are
the distribution of xt conditional on xt−1 and the distribution of zt conditional on zt−1 , respectively. Then, the loglikelihood can be computed as follows:
log pX t (xt |xt−1 , at−1 ; θ) = log pZ t ,Z t−1 (zt , zt−1 |at−1 , θ)
− log pZ t−1 (zt−1 ; θ) + log

∂zt
∂xt

Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

5

The Objective Function: The parameter θ of the model is
optimized to maximize the log-likelihood as in Eqn. (6).
θ ∗ = arg max log pX t (xt |xt−1 , at−1 ; θ)
θ

s.t. zt−1
= Wa at−1 distributes as a Gaussian
a

(6)

This constraint is then incorporated to the objective function
, Σat−1 ; at−1 )
θ ∗ = arg max log pX t (xt |xt−1 , at−1 ; θ) + l(µt−1
a
θ

t−1
t−1
where l(µt−1
) is the log-likelihood
a , Σa ; a
t−1
t−1
a
given mean µa and covariance Σat−1 .

function of

3.2 IRL Learning from Aging Sequence

Fig. 3: Synthesized results with different values of aging
controller. Given an input image, by varying the values of
aging controller, different age-progressed images can be
obtained. Notice that the age controller can help to efficiently control the amount to aging features to be embedded while maintaining other variations between synthesized
faces. Best viewed in color.

The Joint Distributions: In order to model the aging transformation flow, the Gaussian distribution is presented as the
prior distribution pZ for the latent space. After mapping to
the latent space, the age controller variables are also con=
strained as a Gaussian distribution. In particular, let zt−1
a
Wa at−1 represent the latent variables of at−1 . The latent
variables {zt−1 , z̄t , zt−1
a } distribute as Gaussians with means
t−1
{µt−1 , µ̄t , µt−1
, Σ̄ t , Σat−1 } respeca } and covariances {Σ
t
tively. Then, the latent z is as,

In this section, we further extend the capability of our model
by defining an Subject-dependent Deep Aging Policy Network to provide a planning aging path for the aging controller. Consequently, the synthesized sequence, i.e. {x1 , . . . , xT },
is guaranteed to be the best choice in the face aging development for a given subject.
Let ζi = {x1i , a1i , . . . , xTi } be the observed age sequence
of the i-th subject and ζ = {ζ1 , ζ2 , . . . , ζM } be the set of
all M aging sequences in the dataset. The probability of a
sequence ζi can be defined as
P (ζi ) =

1
exp(−EΓ (ζi ))
Z

(7)

whereP
EΓ (ζi ) is an energy function parameterized by Γ , and
Z = ζ̄ exp(−EΓ (ζ̄)) is the partition function computed
using all possible aging sequences ζ̄. Then, the goal is to
learn a model such that the log-likelihood L(ζ; Γ ) of the
observed aging sequences is maximized as follows:
Γ ∗ = arg max L(ζ; Γ ) =
Γ

Y
1
log
P (ζi )
M
ζ ∈ζ

(8)

i

In Eqn. (8), if EΓ (ζi ) is considered as a form of a reward
function, then the problem is equivalent to learning a policy
network from a Reinforcement Learning (RL) system given

zt ∼ N µt , Σ t
a set of demonstrations ζ.

µt = W Wz µt−1 µt−1
+ b + µ̄t
a
The reward function is the key element for policy learn(5)
Σ t = W[Wz Σ t−1 Wz | Σat−1
ing in RL. However, pre-defining a reasonable reward func|
tion for face aging synthesis is impossible in practice. In− (Wz µt−1 )(Wz µt−1 )| µt−1
µt−1
]W|
a
a
deed, it is very hard to measure the goodness of the ageSince the connection between zt−1 and zt embeds the reprogressed images even the ground-truth faces of the subject
lationship between variables of different Gaussian distribuat these ages are available. Therefore, rather than pre-define
tions, we further assume that their joint distribution is also a
Gaussian. Then, the joint distribution pZ t ,Z t−1 (zt , zt−1 |at−1 ; θ) an add-hoc aging reward, the energy EΓ (ζi ) is represented
can be computed as follows.
as a neural network with parameters Γ and adopted as a non t  

linear cost function of an Inverse Reinforcement Learning
t
t,t−1
µ
Σ
Σ
pZ t ,Z t−1 (zt , zt−1 |at−1 ; θ) ∼ N
t−1 ,
t−1,t
t−1
problem.
µ
Σ
Σ
In this IRL system, Γ can be directly learned from the set
t,t−1
z t−1
t−1 |
Σ
= W(W Σ
µa 1 )
of
observed
aging sequences ζ. Fig. 4 illustrates the struc|
Σ t−1,t = (1µt−1
Σ t−1 Wz| )W|
a
ture of the proposed IRL framework. Based on this strucwhere 1 ∈ RD is an all-ones vector.
ture, given a set of aging sequences as demonstrations, not

6

Chi Nhan Duong et al.

Fig. 4: The Subject-Dependent Aging Policy Learning Framework. Given the Age Sequence Demonstrations, the cost function is learned by maximizing the log-likelihood of these sequences. Its output is then can be used to optimize the Subject
Dependent Aging Policy Network which later is able to predict the most appropriate aging path for each subject.
only the cost function can be learned to maximize the loglikelihood of observed age sequence but also the policy, i.e.
predicting aging path for each individual, is obtained with
respect to the optimized cost.
Mathematically, the IRL based age progression procedure can be formulated as follows. Let Mirl {S, A, T , ζ, EΓ }
be a Markov Decision Process (MDP) where {S, A, T } denote the state space, the action space, and the transition model,
respectively. ζ is the set of observed aging sequences and
EΓ represents the cost function. Given an MDP Mirl , our
goal is to discover the unknown cost function EΓ from the
observation ζ as well as simultaneously extract the policy
q(at−1 |xt−1 ) that minimizes the learned cost function.
State: The state st = [xt , age] is defined as a composition of two information, i.e. the face image xt at the t-th
stage; and the age label of xt .
t

Action: Similar to the age controller, an action a is defined as the amount of aging variations that the progression
process should perform on state st . Given st , an action at is
selected by stochastically sampling from the action probability distribution. During testing, given the current state, the
action with the highest probability is chosen for synthesizing
process. Due to data availability where the largest aging distance between the starting and ending images of a sequence
is 15, we choose the length of Na = 16 (i.e. plus one state
of at where st and st+1 has the same age).
Cost Function: The cost function plays a crucial role to
guide the whole system to learn the sequential policies to
obtain a specific aging path for each subject. Getting a state

st and at as inputs, the cost function maps them to a value
cΓ (st , at ). Thus, the cost P
for the i-th aging sequence can
be obtained as EΓ (ζi ) = t cΓ (sti , ati ). In order to learn a
complex and nonlinear cost formulation, each cΓ (st , at ) is
approximated by a neural network with two hidden layers of
32 hidden units followed by Rectified Linear Unit (ReLU).
Policy: Given the cost function cΓ (sti , ati ), the policy is
presented as a Gaussian trajectory distribution as follows.
Y
q(ζi ) = q(s1i )
q(st+1
|sti , ati )q(ati |sti )
(9)
i
t

Then it is optimized respecting to the expected cost function
Eq(ζi ) [EΓ (ζi )].
Given the defined state st and action at , the observation
aging sequence ζi is redefined as ζi = {s1i , a1i , . . . , sTi }. The
log-likelihood L(ζ; Γ ) in Eqn. (8) can be rewritten as,
L(ζ; Γ ) = −

X
1 X
EΓ (ζi ) − log
exp(−EΓ (ζ̄))
M ζ ∈ζ
i

(10)

ζ̄

Since the computation of the partition function is intractable, the sampling-based approach (Finn et al, 2016) is
adopted to approximate the second term of L(ζ; Γ ) in Eqn.
(10) by a set of aging sequences sampled from the distribution q(ζ).
L(ζ; Γ ) ≈ −

1 X
1 X exp(−EΓ (ζj ))
EΓ (ζi ) − log
M ζ ∈ζ
N ζ ∈ζ
q(ζj )
i

j

q

where N is the number of age sequences sampled from a
sampling distribution q. Then the gradient is given by
∇Γ L = −

1 X dEΓ (ζi )
1 X
dEΓ (ζj )
+ 0
wζj
M ζ ∈ζ
dΓ
Z ζ ∈ζ
dΓ
i

j

q

(11)

Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

Algorithm 1 Subject-Dependent Aging Policy Learning
Input: Observed M age sequence ζ = {ζ1 , . . . , ζM } where ζi =
{s1i , a1i , . . . , sT
i }.
Output: optimized cost params Γ and distribution q(ζ)
1
Initialization: Randomly initialize policy distribution qΓ
(ζ) with
a uniform distribution.
for k = 1 to K1 do
k
Sample M aging paths from qΓ
(ζ).
for i = 1 to M do
Apply synthesis component in Section 3.1 given
the i-th sampled aging path and the starting state
s1i to obtain the sampled sequence ζ̄i .
Add ζ̄i to ζqΓk
end for
for i = 1 to K2 do
Sample a batch of observed sequence ζ̂ ⊂ ζ
Sample a batch of sampling sequence ζ̂qΓk ⊂ ζqΓk

7

4 Discussion

The choice of the distribution q is now critical to the
success of the approximation. It can be adaptively optimized
by first initialized with a uniform distribution and followed
an iteratively three-step optimization process: (1) generate a
set of aging sequences ζq ; (2) update the cost function using
Eqn. (11); and (3) refine the distribution q as in Eqn. (12).

By setting up the invertible mapping functions as deep convolutional networks, SDAP structure is able to shares the advantages of its predecessor, i.e. TNVP, in the capabilities of
efficiently capturing highly nonlinear facial features while
maintaining a tractable log-likelihood density estimation.
Besides aging variation, SDAP is also able to effectively
handle other variations such as pose, expression, illumination, etc., as can be seen in Figs. 5 and 6.
Unlike TNVP, SDAP provides a more advanced architecture that optimizes the amount of aging information to
be embedded to input face. This ability benefits not only the
training process, i.e. maximize the training data usability,
but also the testing phase, i.e. flexible and more controlled in
the progressed face to be synthesized via the age controller.
Fig. 3 illustrates different age-progressed results obtained
by varying the values of the age controller. The bigger gap
value produces the older faces.
While previous aging approaches only embed the aging
information via the relationships between the input image
and age label (i.e. direct approach), or images of two consecutive age groups (i.e. step-by-step) approach, SDAP structure aims at learning from the entire age sequence with the
learning objective is designed specifically for sequence (see
Eqn. (7)). Under the IRL framework, the whole sequence
can be fitted into the learning process for network optimization. As a result, more stable aging sequences can be synthesized.

q ∗ = arg min Eq [cΓ (ζ)] − H(ζ)

5 Experimental Results

1:
2:
3:
4:
5:

6:
7:
8:
9:
10:
11:

ζq ← ζ̂ ∪ ζ̂qΓk

12:

Compute the gradient

dL(Γ )
dΓ

using Eqn. (11)

dL(Γ )
dΓ

Update Γ using
end for
k+1
k
(ζ) with ζqΓk and EΓ to qΓ
Update qΓ
(ζ) using approach in
(Levine and Abbeel, 2014).
16: end for

13:
14:
15:

where Z 0 =

q

P

ζj

wζj and wζj =

exp(−EΓ (ζj ))
.
q(ζj )

(12)

5.1 Databases
To solve Eqn. (12), we adopt the optimization approach (Levine
t t
and Abbeel, 2014) that also results in a policy q(a |s ). The
The proposed SDAP approach is trained and evaluated usAlgorithm 1 presents the learning procedure in our policy
ing two training and two testing databases that are not overnetwork and cost function parameters.
lapped. The training sets consist of images from AginG Faces
Face aging with single and multiple inputs: During testing
in the Wild (Duong et al, 2016) and aging sequences from
stage, given a face, its inputs, i.e. image and age, are used
Cross-Age Celebrity Dataset (Chen et al, 2014). In testing,
in the first state s1 . The action for s1 is predicted by the
two common databases, i.e. FG-NET (fgN, 2004) and MORPH,
policy network. Then, the synthesis component can produce
(Ricanek Jr and Tesafaye, 2006) are employed.
the age-progressed face for next state. This step is repeated
AginG Faces in the Wild (AGFW): introduces a largeuntil the age of the synthesized face reaches the target age.
scale dataset with 18,685 images collected from search engines and mugshot images from public domains.
Using this structure, the framework can be easily exCross-Age Celebrity Dataset (CACD) includes 163446
tended to take multiple inputs. Given n inputs to the frameimages of 2000 celebrities with the age range of 14 to 62.
work, they are first ordered by ages and an input sequence
FG-NET is a common testing database for both age esti{s10 , a10 , . . . , sn0 } is created, where si0 denotes the state with
mation and synthesis. It includes 1002 images of 82 subjects
the i-th input face and age; and ai0 is the age difference bewith the age range is from 0 to 69 years old.
tween si0 and si+1
0 . The synthesis component can be emMORPH provides two albums with different scales. The
ployed to obtain the values for latent variable z0n . This varismall-scale
album consists of 1690 images while the largeable can act as “memory” that encodes
all
information
from
 −1 0

1
n
scale one includes 55134 images. We use the small-scale
the inputs. We then initialize s = F2 (zn ), age(s0 ) and
album in our evaluation.
start the synthesis process as in the single input case.

8

Chi Nhan Duong et al.
15

25

+10

10
y/o

12
y/o

+4

11
y/o

+1

+5

+4

+5

+15

+1

+10

+12

+6

+2

+15

+11

+15

+12

+4

+14

+15

+1

+2

+1

+14

+2

+15

60

55
+11

+15

+7

+4

50

+14

+15

+7

20
y/o

45

40

+5

+10

+7

19
y/o

35

30

+13

12
y/o

10
y/o

20

+8

+1

+4

+9

Fig. 5: Age progression results on FG-NET. Given images at different ages and the target age of 60s, SDAP automatically
predicts the optimal aging path and produce plausible age-progressed faces for each subject. Best viewed in color.
5.2 Implementation Details
In order to train our SDAP model, we first extract the face
regions of all images in AGFW and CACD and align them
according to fix positions of two eyes and mouth corners.
Then, we select all possible image pairs (at age t1 and t2 ) of
a subset of 575 subjects from CACD such that t1 ≤ t2 and
obtain 13,667 training pairs. From the images of these subjects, we further construct the observed aging sequence set
by ordering all images of each subject by age. This process
produces 575 aging sequences. Using these training data,
we adopt the structure of mapping functions in (Duong et al,
2017) for our bijections F1 , F2 and pretrain them using all
images from AGFW for the capability of face interpretation.
Then a two-step training process is applied. In the first step,
the structure of synthesis unit with two functions F1 , F2 and
an age controller is employed to learn the aging transformation presented in all 13,667 training pairs. The synthesis
units are then composed to formulate the synthesis component. Then in the second step, the Subject-Dependent Aging
Policy Learning is applied to embed the aging relationships
of observed face sequences and learn the Policy Network.
The structure of F1 , F2 includes 10 mapping units where
each unit is set up with 2 residual CNN blocks with 32 hidden feature maps for its scale and translation components.
The convolution size is 3 × 3. The training batch size is
set to 64. A fully connected network with two layers is employed to build policy model. Each layer contains 32 units.
The training time for our models is 24.75 hours in total on
a machine with a Core i7-6700@3.4GHz CPU, 64.00 GB
RAM and a GPU NVIDIA GTX Titan X. In reinforcement
learning algorithm, rllab (Duan et al, 2016) framework is
used to develop and evaluate our Algorithm 1.

5.3 Age Progression

Since our SDAP is trained using face sequences with age
ranging from 10 to 64 years old, it is evaluated using all
faces above ten years old in FG-NET and MORPH. Given
faces of different subjects, our aging policy can find the optimal aging path to reach the target ages via intermediate
age-progressed steps (Fig. 5). Indeed, SDAP not only produces aging path for each individual, but also well handles
in-the-wild facial variations, e.g. poses, expressions, etc.

In addition, the facial textures and shapes are also naturally and implicitly evolved according to individuals differences. In particular, more changes are focused around the
ages of 20s, 40s and over 50s where beards and wrinkles
naturally appear in the age-progressed faces around those
ages. In Fig. 6, we further compare our synthesized results
against other recent work, including IAAP (KemelmacherShlizerman et al, 2014), CAAE (Zhang et al, 2017), and
TNVP (Duong et al, 2017). The predicted aging path of each
subject is also displayed for reference. When the age distance between the input and target ages becomes large, the
direct age progression approaches usually produce synthesized faces that are similar to the input faces plus wrinkles.
The step-by-step age progression tends to have better synthesis results but still limited in the amount of variations in
synthesized faces. SDAP shows the advantages in the capability of capturing and producing more aging variations in
faces of the same age group. Fig. 7 presents our further results at different ages with the real faces as reference.

Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

Aging Development Path
Input

20

50

40

30

Our
SDAP

9

Step-by-Step Direct Age
Age
Progression
Progression
CAAE IAAP
TNVP

Fig. 6: Comparisons between our SDAP against direct approach, i.e. IAAP (Kemelmacher-Shlizerman et al, 2014), CAAE
(Zhang et al, 2017), and step-by-step approach, i.e. TNVP (Duong et al, 2017). Best viewed in color.

10 y/o

14 y/o

18 y/o

14 y/o

18 y/o

20 y/o

24 y/o

28 y/o

20 y/o

26 y/o

34 y/o

37 y/o

41 y/o

22 y/o

26 y/o

37 y/o

38 y/o

45 y/o

Fig. 7: Age progression using SDAP. Given images (1st column), SDAP synthesizes the subject’s faces at different ages
(row above) against the ground-truth (row below).

5.4 Age Invariant Face Recognition
Our SDAP is also validated using the testing protocol as in
(Duong et al, 2017) with two benchmarking sets of cross-age
face verification. First, we perform small-scale testing protocol by constructing a set A of 1052 randomly picked image pairs from FG-NET with age span larger than 10 years
old. There are 526 positive pairs (the same subjects) and
526 negative pairs (different subjects). For each pair in A,
SDAP synthesizes the face of the younger age to the face of
the older one. This process results in the set B1 . The same

process is then applied using other age progression methods, i.e. IAAP (Tsai et al, 2014), TRBM (Duong et al, 2016)
and TNVP (Duong et al, 2017) to construct B2 , B3 and B4 ,
respectively. Then, the False Rejection Rate-False Acceptance Rate (FRR-FAR) is reported under the Receiver Operating Characteristic (ROC) curves as presented in Fig. 8a.
These results show that our SDAP outperforms other age
progression approaches with a significant improvement rate
for matching performance over the original pairs.
In the large-scale testing protocol, we conduct Megaface
(Kemelmacher-Shlizerman et al, 2016) face verification benchmarking targeted on FG-NET plus one million distractors.
Fig. 8b illustrates how the Rank-1 identification rates change
when the number of distractors increases. The corresponding rates of all comparing methods at one million distractors
are shown in Table 2. Fig. 8c presents the ROC curves respecting to True and False Acceptance Rates (TAR-FAR)
1
. The Sphere Face (SF) model (Liu et al, 2017), trained
solely on a small scale CASIA dataset having < 0.49M images without cross-age information, achieves the best performance among all compared face matching approaches.
Using our SDAP aging model, this face matching model
can achieve even higher matching results in face verification. Moreover, these significant improvements gain without
re-training the SF model and it outperforms other models as
shown in Table 2.

6 Conclusions
This work has presented a novel Generative Probabilistic
Modeling under an IRL approach to age progression. The
1

The results of other methods are provided in MegaFace website.

10

Chi Nhan Duong et al.
ROC curve age span > 10

0.8

1

80

0.8

Identification rate (%)

Pairs B 2 (TNVP)
Pairs B 3 (IAAP)
Pairs B 4 (TRBM)

0.6
0.5
0.4
0.3
0.2
0.1
0
0

0.2

0.4
0.6
False Acceptance Rate

0.8

(a) ROC curves of FG-NET pairs

1

True Acceptance Rate

0.7

False Rejection Rate

100

Pairs A
Pairs B 1 (Ours)

60

40

20
10
10 1

SF + SDAP
SF + TNVP
SF
Deepsense_small
3DiVi-tdvm6
Barebones_FR-cnn
NTechLAB_small

0.6

0.4

0.2

0

10 2

10 3

10 4

10 5

#distractors (log scale)

(b) CMC curves on MegaFace

10 6

10 -5 10 -4 10 -3 10 -2 10 -1
False Acceptance Rate

10 0

(c) ROC curves on MegaFace

Fig. 8: Comparison with other approaches in age invariant face recognition (a) ROC curves of face verification on the
small-scale testing protocol; (b) Cumulative Match Curve (CMC) and (c) ROC curves of SF model (Liu et al, 2017) and its
improvements using age-progressed faces from TNVP (Duong et al, 2017) and our SDAP on the large-scale testing protocol
of MegaFace challenge 1.
Table 2: Comparison results in Rank-1 Identification Accuracy with one million Distractors on MegaFace #1 FG-NET.
Methods
Barebones FR
3DiVi
NTechLAB
DeepSense
SF (Liu et al, 2017)
SF + TNVP
SF + SDAP

Training set
with cross-age faces
with cross-age faces
with cross-age faces
with cross-age faces
without cross-age faces
without cross-age faces
without cross-age faces

Accuracy
7.136 %
15.78 %
29.168 %
43.54 %
52.22%
61.53%
64.4%

model inherits the strengths of both probabilistic graphical
model and recent advances of deep network. Using the proposed tractable log-likelihood objective functions together
deep features, our SDAP produce sharpened and enhanced
skin texture age-progressed faces. In addition, the proposed
SDAP aims at providing a subject-dependent aging path with
the optimal reward. Furthermore, it makes full advantage of
input source by allowing using multiple images to optimize
aging path. The experimental results conducted on various
databases including large-scale Megaface have proven the
robustness and effectiveness of the proposed SDAP model
on both face aging synthesis and cross-age verification

References
(2004) Fg-net aging database. http://www.fgnet.
rsunit.com
Antipov G, Baccouche M, Dugelay JL (2017) Face aging
with conditional generative adversarial networks. arXiv
preprint arXiv:170201983
Burt DM, Perrett DI (1995) Perception of age in adult
caucasian male faces: Computer graphic manipulation of shape and colour information. Proceedings of

the Royal Society of London B: Biological Sciences
259(1355):137–143
Chen BC, Chen CS, Hsu WH (2014) Cross-age reference
coding for age-invariant face recognition and retrieval. In:
ECCV, pp 768–783
Duan Y, Chen X, Houthooft R, Schulman J, Abbeel P (2016)
Benchmarking deep reinforcement learning for continuous control. In: International Conference on Machine
Learning, pp 1329–1338
Duong CN, Luu K, Quach KG, Bui TD (2016) Longitudinal face modeling via temporal deep restricted boltzmann
machines. In: CVPR, pp 5772–5780
Duong CN, Quach KG, Luu K, Le N, Savvides M (2017)
Temporal non-volume preserving approach to facial
age-progression and age-invariant face recognition. In:
The IEEE International Conference on Computer Vision
(ICCV), pp 3755–3763
Finn C, Levine S, Abbeel P (2016) Guided cost learning:
Deep inverse optimal control via policy optimization. In:
International Conference on Machine Learning, pp 49–58
Geng X, Zhou ZH, Smith-Miles K (2007) Automatic
age estimation based on facial aging patterns. PAMI
29(12):2234–2240
Kemelmacher-Shlizerman I, Suwajanakorn S, Seitz SM
(2014) Illumination-aware age progression. In: CVPR,
IEEE, pp 3334–3341
Kemelmacher-Shlizerman I, Seitz SM, Miller D, Brossard
E (2016) The megaface benchmark: 1 million faces for
recognition at scale. In: CVPR, pp 4873–4882
Lanitis A, Taylor CJ, Cootes TF (2002) Toward automatic simulation of aging effects on face images. PAMI
24(4):442–455
Levine S, Abbeel P (2014) Learning neural network policies
with guided policy search under unknown dynamics. In:
Advances in Neural Information Processing Systems, pp

Learning from Longitudinal Face Demonstration - Where Tractable Deep Modeling Meets Inverse Reinforcement Learning

1071–1079
Liu W, Wen Y, Yu Z, Li M, Raj B, Song L (2017)
Sphereface: Deep hypersphere embedding for face recognition. arXiv preprint arXiv:170408063
Luu K, Suen C, Bui T, Ricanek JK (2009) Automatic childface age-progression based on heritability factors of familial faces. In: BIdS, IEEE, pp 1–6
Patterson E, Ricanek K, Albert M, Boone E (2006) Automatic representation of adult aging in facial images. In:
Proc. IASTED Intl Conf. Visualization, Imaging, and Image Processing, pp 171–176
Ricanek Jr K, Tesafaye T (2006) Morph: A longitudinal image database of normal adult age-progression. In: FGR
2006., IEEE, pp 341–345
Rowland D, Perrett D, et al (1995) Manipulating facial appearance through shape and color. Computer Graphics
and Applications, IEEE 15(5):70–76
Shu X, Tang J, Lai H, Liu L, Yan S (2015) Personalized age
progression with aging dictionary. In: ICCV, pp 3970–
3978
Suo J, Zhu SC, Shan S, Chen X (2010) A compositional and
dynamic model for face aging. PAMI 32(3):385–401
Suo J, Chen X, Shan S, Gao W, Dai Q (2012) A concatenational graph evolution aging model. PAMI 34(11):2083–
2096
Taylor GW, Hinton GE (2009) Factored conditional restricted boltzmann machines for modeling motion style.
In: Proceedings of the 26th annual international conference on machine learning, ACM, pp 1025–1032
Tsai MH, Liao YK, Lin IC (2014) Human face aging with
guided prediction and detail synthesis. Multimedia tools
and applications 72(1):801–824
Wang W, Cui Z, Yan Y, Feng J, Yan S, Shu X, Sebe N (2016)
Recurrent face aging. In: CVPR, pp 2378–2386
Yang H, Huang D, Wang Y, Wang H, Tang Y (2016) Face
aging effect simulation using hidden factor analysis joint
sparse representation. TIP 25(6):2493–2507
Zhang Z, Song Y, Qi H (2017) Age progression/regression
by conditional adversarial autoencoder. In: The IEEE
Conference on Computer Vision and Pattern Recognition
(CVPR)

11

