Modeling tagged pedestrian motion: a mean-field type game approach
Alexander Aurell Boualem Djehiche
math.OC 49N90, 49J55, 60H10, 60K30, 91A80, 93E20


equationsection lemmasection








Tagged pedestrian model
Aurell, Djehiche


December 30, 2023




This paper suggests a model for the motion of tagged pedestrians: 
pedestrians moving towards a specified targeted destination, which they are 
forced to reach. It aims to be a decision-making tool for the 
positioning of fire fighters, security personnel and other services in a 
pedestrian environment. Taking interaction with the surrounding crowd into 
account leads to a differential nonzero-sum game model where the tagged 
pedestrians compete with the surrounding crowd of ordinary pedestrians. 
When deciding how to act, pedestrians consider crowd distribution-dependent 
effects, like congestion and crowd aversion. Including such effects in the 
parameters of the game, makes it a mean-field type game. The 
equilibrium control is characterized, and special cases are discussed. Behavior 
in the model is studied by numerical simulations.


MSC 2010: 49N90, 49J55, 60H10, 60K30, 91A80, 93E20

Keywords: pedestrian dynamics; backward-forward stochastic 
differential equations; mean-field type games; congestion; crowd 
aversion; evacuation planning









§ INTRODUCTION


Tagged pedestrians are individuals that plan their motion 
from an unspecified initial position in order to reach a 
specified target location in a certain time. The model for tagged 
pedestrian motion proposed in this paper is based on mean-field type game 
theory, and is a decision making tool for the positioning of fire fighters, 
medical personnel, etc, during mass gatherings.
The tagged's prime objective is the pre-set final destination 
which is considered essential to reach; ending up in proximity of the final 
destination is not acceptable, it has to be reached. This is in sharp 
contrast to the standard finite-horizon 
models cited below, where pedestrians are penalized if their final position 
deviates from a target position, such penalization is a 'soft' constraint and 
can be broken at a cost.
The tagged's initial position is chosen rationally. Therefore, we are 
inclined to think of the tagged as external entities to be deployed in the 
crowd. Where they (rationally) ought to be deployed is subject to an offline 
calculation made by a coordinator: a central planner. Besides tagged 
pedestrian motion, possible applications of the model include cancer cell 
dynamics and smart medicine in the human body, and malware propagation in a 
network, among other.



The central planner's decision making is based on knowledge of 
the pedestrian distribution. As noted in <cit.>, the 
pedestrian behavior in dense crowds is empirically random to some extent, 
likely due to the large number of external inputs. In a noisy environment, the 
central planner anticipates the behavior of the crowd and predicts the tagged's 
path to the target. As is standard in the mean-field approach, interaction with 
tagged and ordinary pedestrians is modeled as reactions to the 
state distribution of a representative tagged and ordinary pedestrian, 
respectively. This leads us to formulate a mean-field type game based
model, which in certain scenarios reduces to an optimal control based model.



 §.§ Related work




  §.§.§ Optimal control and games of mean-field type


Rational pedestrian behavior is, in this paper, characterized by either a game 
equilibrium, or an optimal control. 
The tool used to find the equilibrium/optimal behavior is Pontryagin's 
stochastic maximum 
principle (SMP). For stochastic control problems, SMP yields, when available, 
necessary conditions that must be satisfied by any 
optimal solution. The necessary conditions become sufficient under additional 
convexity conditions. Early results show that an optimal control along 
with the corresponding optimal state trajectory must solve the so-called 
Hamiltonian system, which is a two-point (forward-backward) boundary value 
problem, together with a maximum condition of the so-called Hamiltonian 
function (see <cit.> for a 
detailed account). 
In stochastic differential games, both zero-sum and nonzero-sum, SMP is one of 
the main tools for obtaining conditions for an equilibrium, essentially 
inherited from the theory of stochastic optimal control. For recent examples 
of the use of SMP in stochastic differential game theory, see 
<cit.>.



In stochastic systems, the backward equation is 
fundamentally different from the forward equation, if the solution is required 
to be adapted. An adapted solution to a backward stochastic differential 
equation (BSDE) is a pair of adapted stochastic processes (Y_·, Z_·), 
where Z_· corrects any “non-adaptedness” caused by the terminal 
condition of Y_·. As pointed out in <cit.>, the 
first component Y_· corresponds to the mean evolution of the dynamics, 
and Z_· to the risk between current time and terminal time. Linear BSDEs 
extend to non-linear BSDEs <cit.> and backward-forward SDEs 
(BFSDE) <cit.>. 
BSDEs with distribution-dependent coefficients, mean-field BSDEs, are by now 
well-understood objects <cit.>. 
Mean-field BFSDEs arise naturally in the probabilistic analysis of mean-field 
games (MFG), mean-field type games (MFTG) and optimal control of mean-field 
type equations.



The theory of optimal control of mean-field SDEs, initiated in 
<cit.>, treats stochastic control problems with 
coefficients dependent on the marginal state-distribution. This theory is by 
now well developed for forward stochastic dynamics, i.e., with initial 
conditions on state <cit.>. With SMPs for optimal 
control problems of mean-field type 
at hand, MFTG theory can inherit these techniques, just like 
stochastic differential game theory does in the mean-field free case. See 
<cit.> for a review of solution approaches to MFTGs. 



Optimal control of mean-field BSDEs has recently gained attention. In 
<cit.> the mean-field LQ BSDE control problem with 
deterministic coefficients is studied. Assuming 
the control space is linear, linear perturbation is used to derive a 
stationarity condition which together with a mean-field FBSDE system 
characterizes the optimal control. Other recent work on the control of 
mean-field BSDEs makes use of the SMP approach 
of <cit.> to control of BFSDEs.



Optimal control problems of mean-field type can be interpreted as large 
population limits of cooperative games, 
where the players collaborate to optimize a joint objective 
<cit.>. A close relative to mean-field type control is 
MFG, a class of non-cooperative stochastic differential 
games, initiated by <cit.> and <cit.> 
independently. For both mean-field type control problems and MFG, the 
games approximated are games between a large number of indistinguishable 
(anonymous) players, interacting weakly through a mean-field coupling term.
Weak player-to-player interaction through the mean-field coupling restricts the
influence one player has on any other player to be inversely proportional to 
the number of players, hence the level of influence of any specific player is 
very small. In contrast to the MFG, players in a MFTG can be influential, and 
distinguishable (non-anonymous). That is, state dynamics and/or cost need not 
be of the same form over the whole player population, and a single player can 
have a major influence on other players' dynamics and/or cost.



  §.§.§ Pedestrian crowd modeling

There is a variety of mathematical approaches to the 
modeling of pedestrian crowd motion. Microscopic 
force-based models <cit.>, and in 
particular the social force model, represent pedestrian behavior as a reaction 
to forces and potentials, applied not only by the surrounding environment but 
also by the pedestrian's internal motivation and desire. A cellular 
automata approach to microscopic modeling of pedestrian 
crowds can be found in <cit.>, to mention only few. 
Macroscopic models view the crowd as a continuum, described by averaged 
quantities such as density and pressure. The Hughes model 
<cit.> 
couples a conservation law, representing the physics of the crowd, with an 
Eikonal equation modeling a common task of the pedestrians. Its variations are 
manifold. Kinetic and other multi-scale models 
<cit.> 
constitute an intermediate step between the micro- and the macro scales.  
Microscopic game and optimal control models for pedestrian 
crowd dynamics, with their relevant continuum limits in 
the form of mean-field games and mean-field type optimal 
control, have gained interest in the last decade. In 
<cit.>, a simplified MFG was used to model 
rational behavior of pedestrian in a crowd. Following Lasry and Lion's paper on 
MFG, <cit.> applied MFG to pedestrian crowd 
motion and <cit.> used MFG to model local congestion effects, 
i.e. the relationship between energy needed to walk/run and local crowd density.
MFG based models have also been used to simulate evacuation of pedestrians
<cit.>.



The mean-field approach rests on an exchangeability 
assumption; pedestrians are anonymous, they may have different paths but one 
individual cannot be distinguished from another. While interesting when 
modeling circumstances where pedestrians can be considered indistinguishable, 
for instance a train station during rush hour or fast exits of an area in case 
of an alarm, there are situations where an anonymous crowd model is not 
satisfactory. Mean-field type games is a tool to extend the mean-field 
approach to distinguishable sub-crowds and influential individuals, and was 
applied in <cit.>. Other ways to break the anonymity 
within the mean-field approach are multi-population MFGs 
<cit.> and major 
agent models <cit.>.



Another important characteristic of standard MFGs and MFTGs is the assumption of
anticipative players. Each player is assumed to predict how 
the whole population will act in the future, and then pick a strategy 
accordingly. In a pedestrian crowd setting, this would correspond to pedestrians 
knowing, most likely by experience, how the surrounding crowd will behave. 
This is a very high 'level of rationality', and certainly not appropriate in 
all scenarios. We refer to <cit.> for a precise 
discussion on 
the level of rationality of pedestrian crowd models, including mean-field games 
and optimal control of mean-field type based models.



 §.§ Paper contribution and outline


This paper investigates a new modeling approach to the motion of 
pedestrians whose primary objective is to reach a specific target, at a 
specific time. The model can represent a large group that is steered by a
central planner, or a single individual. Even though these pedestrians have 
certain non-standard goals, they are constrained by the same physical 
limitations as ordinary pedestrians, and the central planner takes this into 
consideration. Moreover, the central planner has access to complete information 
of the surrounding environment, and utilizes it in the decision making process.



The contribution of this paper is a mean-field type game based model for the 
motion of tagged pedestrians in a surrounding crowd of 'ordinary pedestrians'. 
The players in the game are crowds, and act under 
general distribution-dependent dynamics and cost. The tagged pedestrians have a 
'hard' terminal condition, while the ordinary pedestrians have a 'hard' initial 
condition, and this results in a state equation in the form of a mean-field 
BFSDE, representing i) the predicted 
motion of the tagged towards the target, coupled with ii) the evolution of the 
surrounding crowd from a known initial configuration.



Rational behavior in the model is characterized with a version of SMP, tailored 
for the mean-field BFSDE system with mean-field type costs. The central section 
of this paper is the solved examples, where we illustrate pedestrian behavior 
in the model. Further directions of research are also outlined.



The tagged pedestrian model is presented in Section <ref>, which 
begins in a deterministic setting, to which we gradually add components until 
the full model is reached. The SMP that gives necessary and sufficient 
conditions for a pair of equilibrium controls for the mean-field type game is 
presented in Theorem <ref>. Examples of tagged motion are studied 
in Section <ref>. All technical proofs and background theory 
are moved to the appendix.



§ THE TAGGED PEDESTRIAN MODEL


In this section the tagged pedestrian model is introduced. Velocity fields 
are the driving components in the model and include both small-scale pedestrian 
interactions and path planning components. The latter is implemented by 
pedestrians in a rational way: A cost functional summarizing pedestrian 
preferences is minimized. The former 
describes involuntary movement over which the pedestrian has no control. 


 §.§ List of symbols


 
   T∈ (0.∞) – the time horizon.
 
   (Ω, ℱ, 𝔽, ℙ) – the underlying 
  filtered probability space.
 
   𝒫(ℝ^d) – the space of probability measures on 
(ℝ^d, ℬ(ℝ^d)).
 
   𝒫_2(ℝ^d) – all square-integrable measures in 
𝒫().
 
   X∈𝒫(ℝ^k) – the distribution of the 
random variable X∈ℝ^k under ℙ.
 
   X_· – the stochastic process {X_t;  t≥ 0}.
 
   L^2_ℱ() – the space of ℱ-measurable 
square-integrable -valued random variables.





 §.§ Pedestrian dynamics


In a deterministic setting pedestrian state dynamics are 
described by ordinary differential equations (ODE). An ordinary pedestrian is 
initiated at some location x_0∈, and moves according to an 
ODE with an initial condition,

    d/dtX_t = b^x_t,     t ∈ [0,T],
     
    
     X_0 = x_0.

The pedestrian influences its velocity through a control function, 
u^x_·. The control is assumed to take values in the set 
U^x⊂ℝ^d_x, d_x≥ 1. Alongside the control function, 
b^x_t may depend on interaction with other pedestrians, for example through 
collisions. In the literature, the velocity is often split into a 
behavioral velocity (the control) and an interaction velocity,

    b^x_t = u^x_t + b^x,int_t.

So on top of any interaction 
velocity the tagged influences its movement through a control, and this grants 
it some smartness. As was discussed in the introduction, the pedestrian may 
foresee crowd movement and act in advance to avoid congested areas and other 
obstacles. Pedestrian models that considers the behavioral velocity to be an 
internal choice of the pedestrian leaves the framework of classical particle 
models and enters decision-based smart models. A summary of 
the difference between these model classes is found in 
<cit.>.
Alongside ordinary pedestrians, tagged pedestrians are deployed. 
They represent a person on a mission, who has to reach a target location y_T 
∈ at time t=T. In the deterministic setting, the tagged moves according to 
an ODE with terminal 
condition,

    d/dtY_t = u^y_t + b^y,int_t,    t∈[0,T], u^y_t ∈ 
    U^y⊂ℝ^d_y, d_y≥ 1,
      
    
      Y_T = y_T.

Just like u_·^x influences the terminal position of an ordinary 
pedestrian, so does u_·^y influence the initial position of a tagged 
pedestrian. This should be 
interpreted in the following way: the initial position of the tagged pedestrian 
is not pre-determined, but depends on the pedestrian's choice of behavioral 
velocity. This 
choice is subject to the terminal condition, and at the 
same time it adheres to other preferences. For example, if there is a high 
risk 
of injury at a certain location y_T (doors, stairs, etc.), where is the best 
spot for a medic to be positioned, so that she can reach y_T in time T?
Certainly not at y_T, since it is a high risk area. The medic's initial 
location is preferably a safe spot, from which it is easy for her to 
access y_T, taking surrounding pedestrians and environment into account. This 
is what should be reflected in the choice of u_·^y. 



Pedestrian motion can be considered deterministic if the crowd is sparse, but 
partially random if the crowd is dense. 
To capture this, (<ref>)-(<ref>) is extended 
to its stochastic counterpart. Let
(Ω, , ) be a complete probability space, endowed with the filtration 
= (_t)_t≥ 0, 
satisfying the usual conditions. Let the space carry B^x_· and 
B^y_·, independent w_x- and w_y-dimensional -Wiener processes, 
and a random variable y_T∈ L^2__T() independent of B_· := 
(B^x_·, B^y_·)^*. The Brownian motion B_· is split into 
B^x_· and B^y_· to emphasize modeling features: B^x_· is the 
noise that explicitly effects the ordinary pedestrians diffusion, while 
B^y_· may be used to model any noise that in addition to 
B^x_· effects the tagged. All information in the model up to time t is 
contained in _t, and a 
process that depends only on past and current information is called an 
ℱ_t-adapted process. It is natural to require pedestrian 
motion to be adapted, since pedestrians 
react causally to the environment.  In this random environment, we consider a 
control to be feasible if it is open-loop adapted and square integrable, i.e. 
belongs to the sets 𝒰^x and 𝒰^y for ordinary and 
tagged pedestrians respectively,

    𝒰^x    := { u : Ω× [0,T] → U^x | 
    u_· is 𝔽-adapted, 𝔼[∫_0^T |u_s|^2 
    ds] < ∞},
     
    𝒰^y    := { u : Ω× [0,T] → U^y | 
    u_· is 𝔽-adapted, 𝔼[∫_0^T |u_s|^2 
    ds] < 
    ∞}.

As a demonstration of how randomness effects the tagged model, 
consider the tagged dynamics with Brownian small-scale interactions and a 
random terminal condition y_T∈ L^2_ℱ_T(),

    dY_t = (u^y_t + B^y_t)dt,    ∀ t∈[0,T], u^y_·∈𝒰^y,
     
    
     Y_T = y_T.

The naive solution Y_t = y_T 
- ∫_t^T (u^y_s + B^y_s) ds is not _t-adapted, it depends on (B^y_s; 
t≤ s ≤ T)! On the other hand, 

    Y_t = 
    y_T - 
    ∫_t^T u^y_s + B^y_s ds | _t

is _t-adapted. If Y_t from 
(<ref>) is square-integrable for all 
t∈[0,T], which it is in the current setup,
the Martingale Representation Theorem grants
existence of a unique square-integrable 
ℝ^d × (w_x 
+ w_y)-valued and _t-adapted process Z_· such that

    Y_t - ∫_0^t (u^y_s + B^y_s)ds = ∫_0^t Z_sdB_s,   ∀ t∈[0,T],
     
    
     Y_T = y_T.
 
Equation (<ref>) constitutes a 
BSDE. The conditional expectation (<ref>) can be 
interpreted as the 
L^2-projection of the tagged's future path onto currently available 
information. Therefore, a practical interpretation of Z_· is that it is a 
supplementary control used by the tagged, to make its path to y_T the 'best 
prediction' at any instant in time. From a modeling point 
of 
view, the tagged pedestrian thus uses two control processes:



 
  * u^y_· - to heed preferences on initial position, speed, congestion 
and more. It is the tagged's subjective best response to the environment.
  
  * Z_· - to predict the best path to y_T given 
u^y_·. It is a square-integrable process, implicitly given by the 
Martingale Representation Theorem.

Interaction between pedestrians at time t is introduced via the mean-field 
of ordinary pedestrians, X_t := ∘(X_t)^-1, and 
tagged pedestrians, Y_t. They approximate the over-all 
behavior of the crowds in the 
large population limit, under the assumption that within each of the two 
groups individuals are indistinguishable (anonymous). This assumption is in 
place throughout the paper. An example of a mean-field dependent preference 
is the following: to safely accomplish its mission, a security team prefers 
that no individual deviates too far away from the mean position of the team. 
Also, effects like congestion, the extra effort needed when 
moving in a high density area, and aversion, repulsion from other pedestrians, 
can be captured with distribution dependent coefficients. 



In a random environment, with mean-field interactions, we formulate the 
dynamics of representative group members (ordinary and tagged) in the model as

    {
       dY_t    = b^y(t, Θ^y_t, Z_t, Θ^x_t)dt + Z_ 
    tdB_t,
     
    
     dX_t    = b^x(t, Θ^y_t, Z_t, Θ^x_t)dt + 
    σ^x(t, θ^y_t, Z_t, θ^x_t)dB_t^x,
      
    
      Y_T    = y_T,     X_0 = x_0,
      .

where Θ^y_t := (Y_t, Y_t, u^y_t), θ^y_t := (Y_t, 
Y_t) and

    b^x, b^y    : Ω× [0,T] ××𝒫() × U^y 
    ×ℝ^d×(w_x+w_y)××𝒫() × 
    U^x →ℝ,
    
    σ^x    : Ω× [0,T] ××𝒫()×ℝ^d×(w_x+w_y)××𝒫() →ℝ.

Θ^x_t and θ_t^x are defined correspondingly.
Distribution-dependence makes (<ref>) a so-called 
controlled mean-field BFSDE. Appendix 
<ref> summarizes results on existence of
unique solutions to mean-field BFSDEs and states assumptions strong enough so 
that there exists a unique solution to (<ref>) given any 
feasible control pair (u^x_·, u^y_·) ∈𝒰^x×𝒰^y. The assumptions are in force throughout this paper.

Fundamental diagrams, that describe the marginal relations between speed, 
density and flow in a crowd, are not necessary in the construction of b^x and 
b^y, as 
functions of crowd density. It is pointed out in <cit.> 
that the use of fundamental diagrams in pedestrian crowd models is an artifact 
from road traffic models, without proper justification in the case of 
two-dimensional flows. Instead, the more natural (for the purpose of 
modeling pedestrian crowd motion) multidimensional velocity fields b^x and 
b^y are used here.





 §.§ Pedestrian preferences


Modeling pedestrian preferences is a delicate task. Not only is 
gathering of and calibration to 
empirical data difficult for many reasons, but different setups lead to vastly 
different mathematical formulations of rationality. The focus in this paper is 
on setups where pedestrian groups are controlled by a central planner. 
Other possible setups are discussed in Section <ref>.



The ordinary and tagged pedestrians pay the cost f^x and f^y per time unit, 
respectively,

    f^x, f^y : Ω× [0,T] ××𝒫() × U^y 
    ×ℝ^d×(w_x+w_y)××𝒫() × 
    U^x →ℝ.

On top of this, ordinary pedestrians pay a terminal cost h^x at time T, 
and tagged pedestrians pay an initial cost h^y at time 0,

    h^x, h^y : Ω××𝒫() ××𝒫() →ℝ.

Given a control feasible pair (u^x_·, u^y_·) the total cost is J^x : 
𝒰^x →ℝ for the 
ordinary pedestrian, and J^y: 
𝒰^y →ℝ for the tagged,

    J^x(u^x_·; u^y_·)    := 𝔼[∫_0^T 
    f^x(t,Θ^y,Z_t,Θ^x_t)dt + h^x(θ^y_T, θ^x_T) ],
    
    
     J^y(u^y_·; u^x_·)    := 𝔼[∫_0^T 
    f^y(t,Θ^y,Z_t,Θ^x_t)dt + h^y(θ^y_0, θ^x_0) ].



  §.§.§ Mean-field type game

Consider the following situation. Within each crowd, pedestrians 
cooperate, but on a group-level, the crowds compete. This 
constitutes a so-called mean-field type game between the crowds. A 
Nash equilibrium in the game is a pair of feasible controls, (û^x_·, 
û^y_·) ∈𝒰^x ×𝒰^y, satisfying the inequalities

    J^x(u_·; û^y_·) ≥ J^x(û^x_·; 
    û^y_·),   ∀  u_·∈𝒰^x,
      
    
      J^y(u_·; û^x_·) ≥ J^y(û^y_·; 
    û^x_·),   ∀  u_·∈𝒰^y.

The next result is a Pontryagin's type stochastic maximum principle, and yields 
necessary and sufficient conditions for any 
control pair satisfying (<ref>). A proof is provided in 
Appendix <ref>.


Suppose that (û^x_·, û^y_·) is a Nash 
equilibrium, i.e. satisfies (<ref>), and let the regularity 
assumptions of Lemma <ref> be in force. Denote the corresponding 
state processes, given by (<ref>), by X̂_· and 
(Ŷ_·, Ẑ_·) respectively.  
Let (p^xx_·, q^xx_·, q^xy_·), p^xy_·,  
p^yy_· and (p^yx_·, q^yx_·, q^yy_·) solve the 
adjoint equations

    {
      dp^xx_t    = -{∂_x Ĥ^x_t + 
    𝔼[^*(∂_μ^xĤ^x_t)]}dt + 
    q^xx_tdB^x_t + q^xy_tdB^y_t,
      
    
      dp^xy_t    = -{∂_y Ĥ^x_t + 
    𝔼[^*(∂_μ^yĤ^x_t]}dt - ∂_z 
    Ĥ^x_tdB_t,
    
    
    dp^yy_t    = - {∂_y Ĥ^y_t + 
    𝔼[^*(∂_μ^yĤ^y_t)]}dt - 
    ∂_zĤ^y_tdB_t,
    
    
    dp^yx_t    = -{∂_xĤ^y_t + 
    𝔼[^*(∂_μ^xĤ^y_t)]}dt + 
    q^yx_tdB^x_t + q^yy_tdB^y_t,
    
    
    p^xx_T    = -{∂_x ĥ^x + 
    𝔼[^*(∂_μ^xĥ^x)]},   p^xy_0 = 
    0,
    
    
    p^yy_0    = ∂_y ĥ^y + 
    𝔼[^*(∂_μ^yĥ^y)],    
    p^yx_T = 0,
     .

where H^i, i∈{x,y}, is the Hamiltonian, defined by

    H^i(ω, t, y, μ^y, v, z, x, μ^x, u, p^ix, p^iy, q^ix) := 
    ∑_j ∈{x,y} b^j(ω, t, y, μ^y, v, z, x, μ^x, u)p^ij
     
     + σ^x(ω,t, y, μ^y, x, μ^x)q^ix
     -f^i(ω, t, y, μ^y, v, z, x, μ^x, u).

Then

    û^x_t 
       = 
    v∈ U^xargmax H^x(t, 
    Θ̂^y_t, Ẑ_t, θ̂^x_t,
    v, p^xx_t, p^xy_t, q^xx_t),    a.e.-t, ℙ-a.s.
      
    û^y_t 
         =
      v∈ U^yargmax H^y(t, 
    θ̂^y_t, v, Ẑ_t, Θ̂^x_t, p^yx_t, p^yy_t, 
    q^yx_t),    a.e.-t, ℙ-a.s.

If furthermore H^i is concave 
in (y,μ^y, v, z, x, μ^x, u), and h^i is convex in 
(y,μ^y, x, μ^x, z), i∈{x,y}, then any feasible control pair 
satisfying (<ref>) is a Nash equilibrium in the 
mean-field type game.


 Note that in the mean-field type game, the tagged can be thought of as a 
major 
player, influencing the ordinary crowd. Furthermore, the central 
planner has access to a model of the ordinary crowd and this is necessary for 
the determination of the equilibrium control.





  §.§.§ Optimal control of mean-field type

If the central planner does not have access to a model of the crowd of 
ordinary 
pedestrians, any interaction with the surrounding environment will then 
enter the tagged model as a random signal. This is covered by 
the (ω,t)-dependence of b^y and f^y. The mean-field type game 
then reduces to a so-called optimal control problem of mean-field type,

    {u^y_·∈𝒰^ymin    𝔼[∫_0^T 
    f^y(t, Θ^y_t, Z_t)dt + h^y(θ^y_0, Z_0)],
      
    s.t.     dY_t = b^y(t, Θ^y_t, Z_t)dt + Z_ tdB_t,
      
         Y_T = y_T.
      .

Problem (<ref>) is a special case of 
(<ref>) and necessary and sufficient conditions follow as a 
corollary to Theorem <ref>.


Suppose that û^y_· solves (<ref>) 
and denote the corresponding tagged state (Ŷ_·, Ẑ_·). Let 
p_· solve the adjoint equation

    {
      dp_t    = -{∂_yℋ(t, Θ̂^y_t, Ẑ_t, p_t) 
    + 𝔼[^*(∂_μℋ(t, Θ̂^y_t, Ẑ_t, p_t)]}dt
    
         
    - ∂_z ℋ(t, Θ̂^y_t, Ẑ_t, p_t) dB_t,
    
    
    p_0    = ∂_yh^y(θ̂^y_0) + 𝔼[^*(∂_μ 
    h^y(θ̂^y_0)],
     .

where ℋ is the Hamiltonian

    ℋ(ω,t,y,μ,v,z,p) := b^y(ω,t,y,μ,v,z)p - 
    f^y(t,y,μ,v,z).

Then

    û^y_t = v∈ U^yargmax ℋ(t, 
    θ̂^y_t, v, Ẑ_t, p_t),    a.e.-t, ℙ-a.s.

If moreover ℋ is concave in (y,μ, v, z) and that h is 
convex in 
(y,μ), then any feasible û^y_· that satisfies 
(<ref>), almost surely for 
a.e. 
t, is an optimal control to (<ref>).




§ SIMULATIONS



This section is devoted to numerical simulations of the tagged model 
under various external influences and internal preferences. First, two 
scenarios in the optimal control version of the model are considered, including 
preferences on velocity, avoidance and interaction via the mean position of the 
group. Secondly, asymmetric bidirectional flow is simulated with the 
full mean-field type game version of the model. None of 
the parameters used in the simulations stem from real world measurements, but 
velocity profiles  and the asymmetric bidirectional flow are compared 
qualitatively with the experimental studies
<cit.> and <cit.>. 



 §.§ Optimal control: keeping a tagged group together



In this scenario, the common goal of the tagged pedestrians is to stay close to 
the group mean, while conserving energy and initiating in the proximity of 
y_0∈ℝ^2. Distance to the group mean is one of the simplest 
mean-field effects that can be considered. Nonetheless it is a 
distribution-dependent quantity, and the control problem characterizing tagged 
pedestrian behavior in this scenario is the nonstandard optimization problem 
(<ref>). The components of (<ref>) are presented in 
Table <ref>. There is no surrounding crowd present, only 
tagged pedestrians occupy the space. 

    {u_·∈𝒰min    1/2∫_0^T (
    λ_cont(u_t^y)^2 + λ_attr(Y_t - 
    𝔼[Y_t])^2 ) dt + 
    λ_init(Y_0 - 
    y_0)^2,
      
    s.t.     dY_t = (u_t^y + λ_noiseB_t)dt + Z_ tdB_t,
      
         Y_T = y_T.
      .








The scenario is simulated for two sets of parameters, 
see Table <ref>. The mean-field BFSDE system of 
equations characterizing optimal behavior, given by 
Corollary <ref>, is solved with the least-square Monte Carlo 
method of <cit.>. The result is presented in Figure <ref>. 
The group walks approximately on the straight line from the starting area to 
the target point. Remember that the initial position of the tagged is chosen 
rationally by solving (<ref>). There is a trade-off between starting 
close to y_0 and walking with high speed, and the groups rationally initiates 
not at y_0, but somewhere between y_0 and y_T. The group that prefers 
proximity to other group members does indeed move in a more compact formation. 
The difference appears clearly when looking at the mean distance-to-mean of the 
tagged group, see Figure <ref>.











 §.§ Optimal control: desired velocity


Linear-quadratic scenarios have accessible closed form solutions by the method 
of matching. We want to mention the case where the tagged's goal is to move at 
its desired velocity v_des, similar to what was originally 
introduced as the desired speed and direction in 
<cit.>. This is an important special case, since desired 
velocity is measurable in live experiments. See for example
<cit.> for the speed profile of pedestrian walking in 
a straight corridor, starting from standing still. 
The scenario is formulated as standard optimal control problem, 
(<ref>), and 
the setting is summarized in Table <ref>. 

    {u ∈𝒰[0,T]min    1/2∫_0^T (
    λ_cont(u_t^y)^2 + λ_des(u_t^y - 
    v_des(t))^2 + λ_rep(Y_t - 
    Q)^2) dt,
      
    s.t.     dY_t = (u^y_t + λ_noiseB^y_t)dt + Z_ tdB_t,
      
         Y_T = y_T.
      .


 In view of Corollary <ref>, the optimal control is

    û^y_t = p_t + 
    λ_desv_des(t)/λ_cont + 
    λ_rep,

With the ansatz Ŷ_t = γ(t)p_t + η(t)B^y_t + θ_t, 
γ(T) 
= η(T) = 0 and θ(T) = y_T, a matching argument gives the optimally 
controlled dynamics up to a system of ODEs:

    d/dtγ(t) = -λ_repγ(t)^2 + 
    1/λ_cont + 
    λ_des,    γ(T) = 0,
      
    d/dtη(t) = - λ_repγ(t)η(t) + 
    λ_noise,    η(T) 
    = 0,
      
    d/dtθ(t) = -λ_repγ(t)(θ(t) - Q) + 
    λ_desv_
    des(t)/λ_cont+λ_des , 
       θ(T) = 
    y_T,
     
    Ẑ_t = η(t).

In Figure 3, the simulated tagged crowd density is presented for two 
values of 
Q. The desired velocity is set to be negative in both directions for 
t∈[0,T/2], and positive for t ∈(T/2, T], which corresponds to a 
preference to first move south-west during the first half of the time period 
and then turn around and move north-east. The parameter λ_rep 
is set to a negative value, hence the tagged prefers to avoid Q. Parameters 
used in the simulation are summarized in Table <ref>.
The trade-off between walking in the desired velocity and walking close to 
the diamond Q before reaching the target circle is clearly visible.  Recall 
that the initial position is determined by the optimization procedure. In 
this scenario, there is no preference on initial position and the 
tagged group compensates for the location of Q by changing its initial 
position!



In <cit.> the average time-dependent 
velocity of a pedestrian initially standing still is measured experimentally in 
the absence of interactions. The result is a relationship between speed and 
time, than can be used as data for v_des. Approximating the graph 
presented in <cit.> with 

    v_des(t) = max{0.1, arctan(π t - 1.6)},

the scenario is simulated with the two presented in 
Table <ref>. The result is presented in 
Figure 4. The parameter set with a higher penalty on from deviation from 
desired velocity naturally results in a velocity profile closer to 
v_des(t).


















 §.§ Mean-field type game: asymmetric bidirectional flow


Consider now a scenario where ordinary pedestrians initiate at x_0∈ℝ^2, close y_T∈ℝ^2, the location of an incident. They 
begin to walk towards the safe spot x_T∈ℝ^2. A tagged pedestrian is 
to end up at the location of the incident y_T at time t=T. The tagged 
pedestrian is repelled by the mean of the ordinary pedestrian crowd, while the 
ordinary pedestrian crowd is repelled by the tagged pedestrian. This scenario 
is implemented as the MFTG (<ref>), summarized in 
Table <ref>.

    J^x(u^x_·; u^y_·) = 1/2𝔼[∫_0^T (
    λ^x_cont(u^x_t)^2 + 
    λ^x_rep(X_t-Y_t)^2)dt + 
    λ^x_term(X_T-x_T)^2  ],
      
    
      dX_t = u_t^xdt + σ dB^x_t,  σ∈ℝ,   X_0 = x_0,
      
    
      J^y(u^y_·; u^x_·) = 1/2𝔼[∫_0^T 
    (λ^y_cont(u^y_t)^2 + λ^y_red(Y_t 
    - 𝔼[X_t])^2)dt + λ^y_init(Y_0-y_0)^2 ],
      
    
      dY_t = (u_t^y + λ^y_noisedB^y_t) + Z_t dB_t.




In Figure 5 and Figure 6 the scenario is simulated for the parameter sets 
presented in Table <ref>. In Figure 5, the ordinary and the 
tagged do not have to cross paths to go from their initial to their 
terminal positions. The simulated paths (top plot of Figure 5) are similar in 
shape to both the outcome of the corridor experiment under 'condition 3' (no 
obstacle) of <cit.> and the BFR-SSL experiment of 
<cit.>. These experimental studies were conducted in a 
controlled environment that is outside the tagged model presented 
in this paper. Anyhow, the tagged model replicates the separation of lanes in a 
bidirectional pedestrian flow and the uncertainty that pedestrian motion 
exhibits. In the 
density snapshots (bottom row of Figure 5) reveal that in simulated scenario, 
the tagged tagged moves in almost constant velocity towards y_T, while the 
ordinary group lingers a while at x_0 before it starts to move towards x_T.
In Figure 6, the tagged's and the ordinary's straight path from initiate 
position to target cross each other. In this scenario, the ordinary pedestrians 
resolve this by taking walking in a half-circle around the tagged, before 
moving towards their preferred terminal position x_T.











§ CONCLUDING REMARKS AND RESEARCH PERSPECTIVES


A mean-field type game model for so-called tagged pedestrian motion has been 
presented and the reliability of the model has been studied through 
simulations. To 
perform simulations, necessary and sufficient conditions for 
a Nash equilibrium are provided in Theorem <ref>. The theorem is 
proven under quite restrictive conditions on involved coefficient functions. 
However, necessary conditions for a Nash equilibrium in similar games are 
available under less restrictive conditions and since our proof follows a 
standard path, the conditions can certainly be relaxed. The model captures both 
game-like and minor agent-type scenarios. In the 
latter, the tagged cannot effect crowd movement while in the 
former, the tagged and the surrounding crowd have conflicting interests, 
interact, and compete. The rational pedestrian behavior in the competative 
game-like scenario is to use an equilibrium strategy.



There are many variations to the mean-field type game approach. The scenarios 
that have been considered in this paper fall into two rather extreme 
categories; our pedestrians have acted under either basic or 
optimal rationality, in the terms of <cit.>. 
When pedestrians neither have access to information about their 
surroundings or the ability to anticipate pedestrian behavior, the best they 
can do is to implement a control policy based on their own 
position and target position. This is a basic level of rationality. If the 
full model is available and pedestrians cooperate, they can implement a 
control policy of optimal rationality. If the tagged can 
observe crowd densities at each instant in time but not anticipate 
future movement, dynamic pedestrian preference may be modeled as a set of 
control problems: for each 
τ∈[0,T],

    {u^y_.(τ) ∈𝒰^y_τmin    ∫_τ^T 
    f^y(t, 
    Y_t, 
    Y_τ, u^y_t(τ))dt + 𝕀{τ = 0}h^y(Y_0, Y_0),
      
    s.t.     dY_t = b^y(t, Y_t, Y_τ, Z_t, u^y_t(τ))dt + Z_ 
    tdB_t,
      
         Y_T = y_T,
      .

where 𝒰^y_τ is defined in the same way as 𝒰^y, but 
with the interval [0,T] replaced by [τ, T], cf. (<ref>).
This is an intermediate level between basic and optimal rationality. Pedestrian 
decision making can also be modeled as a decentralized mechanism, i.e. instead 
of cooperating, pedestrians compete in a game-like manner within the crowd. 
Decentralized crowd formation can be modeled by a MFG:

 
  (i) Fix a deterministic function μ_·: [0,T] →𝒫_2().
 
  (ii) Solve the stochastic control problem
 
    {u^y_·∈𝒰^ymin    ∫_0^T fy(t, 
    Y_t, 
    μ_t, u^y_t)dt + h^y(Y_0, μ_0),
      
    s.t.     dY_t = b(t, Y_t, μ_t, Z_t, u^y_t)dt + Z_ tdB_t,
      
         Y_T = y_T.
      .

 
  (iii) Determine the function μ̂_· : [0,T]→𝒫_2() such that μ̂_t is the law of the 
optimally 
controlled state from Step (ii) at time t.

Furthermore, minimal exit time (evacuation) problems can be posed at all levels 
of rationality. 



The numerical simulation of mean-field BFSDE systems is in this paper done 
either with the least-square Monte Carlo method of <cit.>, 
or by reducing them to a system of ODEs by the method of matching. 
The downside with the least-square Monte Carlo method is that it is not clear 
which basis functions to use and the matching method is feasible only for 
linear quadratic problems.
Other simulation approaches include deep learning <cit.> and 
fixed-point schemes <cit.>. Fast and stable numerical 
solvers for mean-field BFSDEs beyond the linear-quadratic case is an area of 
research that would benefit many applied fields. In pedestrian crowd 
modeling, improved solvers would facilitate simulation when effects like 
congestion, crowd aversion, and anisotropic preferences are present.






§ MEAN-FIELD BFSDE


Given a control pair (u^x_·, u^y_·) ∈𝒰^x×𝒰^y, systems of the form (<ref>) have been 
studied in the context of optimal control of mean-field type, where they 
naturally arise as necessary optimality conditions. This appendix summarizes 
some of the results on existence and uniqueness of solutions to MF-BFSDEs. 
Let

    ℍ^2,d := { V_· -valued and prog. meas. 
    : 𝔼[∫_0^T |V_s|^2ds] < ∞}
       𝕊^2,d := { V_· -valued and prog.meas. 
    : 𝔼[sup_s∈[0,T] |V_s|^2 ] < ∞}

and recall that, in the case of a fixed control pair, b^x and b^y are 
functions of

    (ω, t, y, μ^y, z, x, μ^x) ∈Ω×[0,T]××𝒫_2() ×ℝ^d×(w_x+w_y)××𝒫()

and σ^x is a function of

    (ω, y, μ^y, z, x, μ^x) ∈Ω××𝒫_2() ×ℝ^d×(w_x+w_y)××𝒫().




 §.§ Quadratic-type constraints

For t∈[0,T], μ^y, μ^x∈𝒫2(), x,x̅,y,y̅∈ and z,z̅∈ℝ^d× (w_x+w_y), let

    𝒜(t,y, y̅, μ^y, v,z,z̅, x, x̅, μ^x) 
     
       =
     (b^y(t,y,μ^y,v, z, x,μ^x) - b^y(t,y̅,μ^y, v, z̅, 
    x̅, μ^x))·(y-y̅) 
    
          + (b^x(t,y,μ^y,v, z, x,μ^x) - b^x(t,y̅,μ^y, v, z̅, 
    x̅, μ^x))·(x-x̅) 
    
           + (σ^x(t,y,μ^y, z, x,μ^x) - σ^x(t,y̅,μ^y, 
    z̅, x̅, μ^x))·(z-z̅).

In <cit.>, the authors provide conditions on
𝒜 under which, alongside standard assumptions, 
(<ref>) has a unique solution 
(X_·, Y_·, Z_·) in ℍ^2,d×ℍ^2,d×ℍ^2,d×(w_x+w_y) for all T.



 §.§ Small time constraint


Under standard Lipschitz- and linear growth-conditions and for a non-degenerate 
diffusion σ^x, (<ref>) has a unique solution 
(X_·, Y_·, Z_·) ∈𝕊^2,d×𝕊^2,d×ℍ^2,d× (w_x+w_y) for small enough T 
<cit.>. The bound on T depends on the Lipschitz 
coefficients. In <cit.>, the 
authors provide an example where uniqueness fails.




§ DIFFERENTIATION OF MEASURE-VALUED FUNCTIONS


The differentiation of measure-valued functions is handled with the lifting 
technique, introduced by P.-L. Lions and outlined in for example 
<cit.>.
Assume that the underlying probability space is 
rich enough, so that for every μ∈_2() there is a random variable Y ∈ 
L^2_ℱ(Ω; ) such that Y = μ. A 
probability space with this property is ([0,1], ℬ([0,1]), dx).  
Under this assumption, any function f: _2() →ℝ 
induces a function F : L^2() →ℝ so that F(Y) := 
f(Y). The Fréchet derivative of F at Y, whenever it exists, is the 
continuous linear functional DF[Y] that satisfies

    F(Y') - F(Y) = 𝔼[DF[Y]·(Y'-Y)] + o(Y'-Y_2),   ∀ Y'∈ L^2_ℱ(Ω; ).

Riesz' Representation Theorem yields 
uniqueness of DF[Y]. Furthermore, there 
exists a Borel function φ[μ] : ℝ^d →ℝ^d, 
independent of the version of Y, such that DF[Y] = φ[Y](Y)
<cit.>. Therefore

    f(Y') - f(Y) = 𝔼[φ[Y](Y)(Y'-Y)] + 
    o(Y'-Y),   ∀ Y'∈ L^2_ℱ(Ω; ).

Denote ∂_μ f(μ;x) := φ[μ](x), x∈, and 
∂_μ f((Y); Y) =: ∂_μ f((Y)). The following identity 
characterizes derivatives with respect to elements in 𝒫_2(),

    DF[Y] = φ[Y](Y) = ∂_μ f(Y).

Equation (<ref>) is the Taylor approximation of a 
measure-valued function. Consider now an f that besides the 
measure takes another argument, ξ. Then 

    f(ξ, Y') - f(ξ,Y) = 𝔼[ ∂_μ 
    f(ξ, Y; Y)(Y'-Y)] + o(Y-Y'_2),

where the expectation is taken over non-tilded random 
variables. This is abbreviated as

    𝔼[∂_μ f(ξ, Y; Y)(Y'-Y)] =: 
    𝔼[(∂_μ f(ξ, Y))^*(Y'-Y)].

Note that Y is deterministic, so the expectation is only taken over the 
'directional argument' of ∂_μ f, Y. Also, the expected value 
(<ref>) is stochastic, since it is not taken over 
ξ. Taking another expectation and changing the order of integration yields

    𝔼[𝔼[∂_μ f(ξ, 
    Y; Y)] (Y'-Y)],

where the tilded expectation is taken over tilded random variables. 
This is abbreviated as

    E[∂_μ f(ξ, Y; Y)] =: 
    𝔼[ ^*( ∂_μ f(ξ, Y)) ]

Example from Section <ref>
The following measure derivative appears in Section 3.1,

    𝔼[ ^*( ∂_μ(Y_t - ∫_ℝ^2yY_t(dy))^2)].

Note that (Y_t - ∫_ℝ^2yY_t(dy))^2 = 
(Y_t -M)^2 =: F(M) where M is a random variable with 
probability law Y_t. By the Taylor expansion, DF[M] = 
2(Y_t - M) and therefore 

    𝔼[ ^*( ∂_μ(Y_t - ∫_ℝ^2yY_t(dy))^2)] = 2(Y_t 
    - M) = 0.





§ PROOF OF THEOREM <REF>


Let u̅^x,ϵ_· be a spike 
variation of û^x_·,

    u̅^x,ϵ_t :=
    {[          û^x_t, t ∈ [0,T]\ E_ϵ,;            u_t,        t ∈ E_ϵ, ].

where u_·∈𝒰^x and E_ϵ is a subset of 
[0,T] of measure ϵ. Given the control pair 
(u̅^x,ϵ_·, û^y_·), denote the corresponding 
solution to the state equation (<ref>) by 
X̅^ϵ_· and (Y̅^ϵ_·, Z̅^ϵ_·). 
To ease notation, let for ϑ∈{b^x, b^y, σ^x, f^x, f^y, h^x, 
h^y},

    ϑ̅^ϵ_t := ϑ(t, θ̅^y,ϵ_t, 
    û^y_t, Z̅^ϵ_t, Θ̅^x,ϵ_t),
     ϑ̂^ϵ_t := ϑ(t,Θ̂^y_t,Ẑ_t, 
    Θ̂^x_t), 
     δ_x ϑ(t) := ϑ(t, Θ̂^y_t, Ẑ_t, 
    θ̂^x_t, u̅^x,ϵ_t) - ϑ̂^ϵ_t.

Consider the ordinary pedestrian's potential loss, would she switch from the 
equilibrium control û^x_· to the perturbed 
u̅^x,ϵ_·,

    J^x(u̅^x,ϵ_·; û^y_·) - J(û^x_·; 
    û^y_·) 
       = 
    𝔼[∫_0^T (f̅^x,ϵ_t - 
    f̂^x_t)dt +  h̅^x,ϵ_T - 
    ĥ^x_T ].

A Taylor expansion of the terminal cost difference yields

    𝔼[h̅^x,ϵ_T  - ĥ^x_T]
    =
    𝔼[(∂_y ĥ^x_t + 
    𝔼[^*(∂_μĥ^x_T)])(X̅^ϵ_t - X̂_t)]
    + o(X̅^ϵ_T - X̂_T_2),

Let X^x_· and (Y^x_·, 
Z^x_·) be the first order variation processes, solving the 
linear BFSDE system

    {
    dX^x_t    = {(∂_yb̂^x_t + 
    𝔼[^*(∂_μ^yb̂^x_t)])Y^x_t + 
    ∂_zb̂^x_tZ^x_t + (∂_xb̂^x_t +
    𝔼[^*(∂_μ^xb̂^x_t)])X^x_t
    
        + δ_x b^x(t)1_E_ϵ(t)}dt
    
          + {(∂_yσ̂^x_t + 
    𝔼[^*(∂_μ^yσ̂^x_t)])Y^x_t + 
    ∂_zσ̂^x_tZ^x_t + (∂_xσ̂^x_t +
    𝔼[^*(∂_μ^xσ̂^x_t)])X^x_t }
    dB^x_t,
    
    
    dY^x_t    = {(∂_yb̂^y_t + 
    𝔼[^*(∂_μ^yb̂^y_t)])Y^x_t + 
    ∂_zb̂^y_tZ^x_t + (∂_xb̂^y_t +
    𝔼[^*(∂_μ^xb̂^y_t)])X^x_t
    
        + δ_x b^y(t)1_E_ϵ(t)}dt + 
    Z^x_tdB_t,
    
    X^x_0    = 0,   Y^x_T = 0.
    .



Assume that b^x are b^y are Lipschitz in the controls, that b^x, b^y, f^x, 
f^y and σ^x are differentiable at the equilibrium point almost surely 
for all 
t, that their derivatives are bounded almost surely for all t and that

    ∂_xĥ^x + 𝔼[^*(∂_μ^xĥ^x)] 
    ∈ L^2_ℱ_T(),  ∂_yĥ^y + 𝔼[^*(∂_μ^yĥ^y)] 
    ∈ L^2_ℱ_0().
 
Then for some positive constant C,

    sup_t∈[0,T]𝔼[ |X^x_t|^2 + 
    |Y^x_t|^2 + ∫_0^tZ^x_s_F  ds ] ≤ 
    Cϵ^2. 
      
       sup_t∈[0,T]𝔼[ 
    |X̅^ϵ_t - X̂_t - X^x_t|^2 + 
    |Y̅^ϵ_t - Ŷ_t - Y^x_t|^2 + 
    ∫_0^tZ̅^ϵ_s - Ẑ_s - Z^x_s_F  ds ] 
    ≤ 
    Cϵ^2.



 The proof is a combination of standard estimates, see 
<cit.> for the SDE terms and <cit.> for the 
BSDE terms.

The adjoint processes and the 
Hamiltonian, defined in (<ref>) and 
(<ref>) respectively, yield together with 
Lemma <ref> and integration by parts that

    J^x(u̅^x,ϵ_·; û^y_·) - J(û^x_·; 
    û^y_·) 
    = 
    𝔼[∫_0^T {(b̅^x,ϵ_t - 
    b̂^x_t)p^xx_t + (b̅^y,ϵ_t - 
    b̂^y_t)p^xy_t 
    
        + (σ̅^x,ϵ_t - 
    σ̂^x_t)q^xx_t - (H̅^x,ϵ_t - 
    Ĥ^x_t)} dt - p^xx_TX^x_T ] + o(ϵ)
    
       =
    𝔼[∫_0^T {(b̅^x,ϵ_t - 
    b̂^x_t)p^xx_t + (b̅^y,ϵ_t - 
    b̂^y_t)p^xy_t + (σ̅^x,ϵ_t - 
    σ̂^x_t)q^xx_t - (H̅^x,ϵ_t - 
    Ĥ^x_t)} dt
    
         - ∫_0^T X^x_t dp^xx_t - ∫_0^T p^xx_t 
    dX^x_t - ∫_0^T d⟨ p^xx_·, 
    X^x_·⟩_t
    
         - ∫_0^T Y^x_t dp^xy_t - ∫_0^T p^xy_t 
    dY^x_t - ∫_0^T d⟨ p^xy_·, 
    Y^x_·⟩_t ] + o(ϵ)
    
       =
    -𝔼[ ∫_0^T δ_x H^x(t)1_E_ϵ(t)dt ] + 
    o(ϵ),

where H̅^x,ϵ_t, Ĥ^x_t and δ_xH^x(t) are defined in 
line with (<ref>). The final equality is retrieved by 
expanding all differences on the third row of (<ref>), canceling all but 
δ_xH^x(t)1_E_ϵ(t) with the forth and fifth row, while making use 
of the estimates from Lemma <ref>. 



Consider now a spike variation of the tagged's control,

    ǔ^y,ϵ_t :=
    {[          û^y_t, t ∈ [0,T]\ E_ϵ,;            u_t,        t ∈ E_ϵ, ].

where u_·∈𝒰^y. Following the same lines of calculations as 
above, one finds that if p^yx_·, p^yy_· are given by the 
adjoint equations (<ref>) and the Hamiltonian H^y 
by (<ref>), then

    J^y(ǔ^y,ϵ_·; û^x_·) - J^y(û^y_·; 
    û^x_·) = -𝔼[∫_0^T δ_y H^y(t)1_E_ϵ 
    dt] + o(ϵ),

where δ_y H^y(t) is defined in line with (<ref>), 
for the spike perturbation ǔ^y,ϵ_t.
The rest of the proof is standard, and can be found in for example 
<cit.>.


 plain
 


