Cooperation, Correlation and Competition in Ergodic $N$-player Games and Mean-field Games of Singular Controls: A Case Study
Federico Cannerozzi Giorgio Ferrari



 
We consider ergodic symmetric N-player and mean-field games of singular control in both cooperative and competitive settings. The state process dynamics of a representative player follow geometric Brownian motion, controlled additively through a nondecreasing process. Agents aim to maximize a long-time average reward functional with instantaneous profit of power type. The game shows strategic complementarities, in that the marginal profit function is increasing with respect to the dynamic average of the states of the other players, when N<∞, or with respect to the stationary mean of the players' distribution, in the mean-field case. In the mean-field formulation, we explicitly construct the solution to the mean-field control problem associated with central planner optimization, as well as Nash and coarse correlated equilibria (with singular and regular recommendations).
Among our findings, we show that coarse correlated equilibria may exist even when Nash equilibria do not. Additionally, we show that a coarse correlated equilibrium with a regular (absolutely continuous) recommendation can outperform a Nash equilibrium where the equilibrium policy is of reflecting type (thus singularly continuous). Furthermore, we prove that the constructed mean-field control and mean-field equilibria can approximate the cooperative and competitive equilibria, respectively, in the corresponding game with N players when N is sufficiently large.
To the best of our knowledge, this paper is the first to characterize coarse correlated equilibria, construct the explicit solution to an ergodic mean-field control problem, and provide approximation results for the related N-player game in the context of singular control games.



Flexible Field Sizes in Secure Distributed Matrix Multiplication via Efficient Interference Cancellation
    
    Okko Makkonen

    Department of Mathematics and Systems Analysis

    Aalto University

    Espoo, Finland

    Email: okko.makkonen@aalto.fi

    Received / accepted 
========================================================================================================================================================





Keywords: mean-field games; N-player games; singular stochastic control; ergodic reward; Nash equilibrium; Pareto efficiency; coarse correlated equilibrium; strategic complementarities.



MSC2020 subject classification: 91A11, 91A15, 91A16, 49N80.




§ INTRODUCTION


In this paper, we investigate ergodic stochastic games of singular control in both competitive and cooperative settings, considering scenarios with a finite number N of players as well as in the mean-field limit. In the formulation with N players, each symmetric player, indexed by i=1,2,...,N, seeks to maximize a long-term average reward functional. The instantaneous profit at time t ≥ 0 is given by π(X^i_t,θ^N_t)=(X^i_t)^α(θ^N_t)^β, where α, β∈ (0,1). Here, X^i_t represents the current level of the state variable for agent i, and θ^N_t=1/N-1∑_j≠ iX^j_t denotes the empirical average of the state processes of all the other N-1 agents. Each agent i can control the geometric dynamics of X^i by increasing its level through a nondecreasing control process ν^i, with the cost of control being proportional to the effort expended. We introduce the concepts of coarse correlated equilibrium, Nash equilibrium, and Pareto efficiency for this game. The notion of Pareto efficiency is associated with the problem of a central planner who seeks to maximize the average of the rewards of all N agents.

The game under study shows strategic complementarities (since the marginal profit is increasing in its second variable; cf. <cit.>) and finds natural applications in dynamic oligopolies, such as in Cournot oligopoly with complementary products or in advertising games (see, e.g., <cit.>). In this regard, the state variable of each agent can be the output or the goodwill stock, which is increased by irreversible investment or advertising, respectively. The resulting payoff is then derived through an isoelastic inverse demand function, depending on the aggregate level of production or goodwill in the entire market. In particular, the ergodic structure of the reward functional we consider is relevant in the context of investment into public goods, in which it might be important to take care of the payoffs received by successive generations.

Constructing equilibria for N-player games in continuous-time and space is a challenging problem. The theory of mean-field games, developed independently by <cit.> and <cit.>, provides approximation results for equilibria of symmetric games with finite players. Indeed, it is typically possible to prove that mean-field equilibria define ε-equilibria for the related N-player games. In this paper, we introduce the mean-field version of the previously described stochastic game and explicitly construct the mean-field Nash equilibrium and the solution to the mean-field central planner control problem.
We also determine sufficient conditions for the existence of coarse correlated equilibria (based on suitable recommendations of the moderator). In the mean-field game, the representative agent reacts to the long-term average of the distribution of the population, which is represented by a scalar parameter θ. The stationary one-dimensional setting of the mean-field game and control problem allows for explicit characterizations of the equilibria (see also <cit.> and references therein in the context of singular/impulse control games).



Our contributions. Despite the specific setting in which the game is formulated (geometric Brownian dynamics and profit function of power type), the analysis reveals a rich structure of the solution while also requiring technical results and arguments. Among these, we highlight the derivation of novel first-order conditions for optimality in ergodic singular stochastic control problems (see Lemma <ref> below), which are of independent interest, as well as the probabilistic representation of the Lagrange multiplier employed in the analysis of the mean-field central planner control problem (see Lemma <ref>).


Our main results are as follows. First, to the best of our knowledge, this is the first paper that constructs the explicit solution to an ergodic mean-field singular stochastic control problem (Theorem <ref>) and proves that its solution can approximate the solution to a central planner problem aiming to achieve Pareto efficiency in the game with N players (see Theorem <ref>).
We construct the mean-field solution using a Lagrange-multiplier approach, which transforms the original McKean-Vlasov control problem into a two-stage optimization problem, in which one first optimizes over the admissible control variables and then over the mean-field parameter.
It is noteworthy that the probabilistic representation of the Lagrange multiplier as the derivative of the mean-field control problem's value function with respect to the mean-field parameter (see Lemma <ref> and Remark <ref> below) is the key ingredient for suitably applying the Law of Large Numbers and completing the proof of the approximation result in Theorem <ref>.

Secondly, we completely characterize the Nash equilibria in the ergodic mean-field game and prove that their existence and uniqueness depend on the strength of the strategic complementarity, measured by the parameter β∈ (0,1) (see Theorem <ref> below). In particular, if 0 < β < 1-α or 1-α < β < 1, then a unique Nash mean-field equilibrium exists, where the state process is reflected upwards à la Skorohod at an explicitly given barrier. On the other hand, if β=1-α, either infinitely many equilibria exist, each of reflecting type, or none exist. The existence of multiple equilibria is related to the fact that the mean-field game under study faces strategic complementarities (see the seminal <cit.> and also <cit.> for contributions on mean-field games). 

The potential presence of multiple Nash equilibria leads to the question of how players can coordinate towards one of them. As the final main contribution of this paper, we consider coarse correlated equilibria.
The concept of coarse correlated equilibria offers an alternative to Nash equilibria, expanding on the latter by incorporating a mediator, or correlation device, that enables agents to adopt correlated strategies without any cooperation.
We determine sufficient conditions for the existence of coarse correlated equilibria, with singular and regular (absolutely continuous) recommendations for the competitive mean-field game of singular controls (see Propositions <ref> and <ref> below).
Since there conditions are non-linear in moments of the correlation device, and present intricate dependence on the model's parameters, we consider a specific choice of parameters, and through a numerical analysis we show that, in the case in which the correlation device has Gamma distribution, there are infinitely many instances of Gamma distributions under which coarse correlated equilibria may exist even when Nash equilibria do not.
Additionally, we show that a coarse correlated equilibrium with a recommendation in the form of a regular control can outperform the Nash equilibrium, whose equilibrium policy is instead of a reflecting type (and thus singularly continuous), highlighting a feature of coarse correlated equilibria well known in standard game theory literature ( see, e.g., <cit.>).





Related literature. Our paper contributes to various streams of literature. Firstly, we add to the literature on mean-field games with singular stochastic controls, a field where a still limited but rapidly increasing number of contributions has focused on abstract results regarding the existence and uniqueness of equilibria (see <cit.>), as well as on explicit characterizations of the Nash solution (see <cit.>). Particularly relevant to our paper is <cit.>, where, in the context of a mean-field game with singular controls, stationary discounted and ergodic Nash equilibria are explicitly constructed and related via the vanishing discount factor method.

Secondly, we contribute to the literature studying games with strategic complementarities (also known as submodular/supermodular games) and the potential emergence of multiple equilibria. This class of games has garnered significant attention in Economics. As Xavier Vives asserts in <cit.>, "Complementarities are intimately linked to multiple equilibria and have a deep connection with strategic situations, and the concept of strategic complementarity is at the center stage of game-theoretic analyses." Among the myriad contributions, we refer to the deterministic games considered in <cit.>, as well as to the dynamic stochastic formulations presented in <cit.> and references therein. To the best of our knowledge, our paper is the first to provide a comprehensive analysis of a stationary mean-field singular stochastic game with strategic complementarities by explicitly characterizing its Pareto efficient outcome, as well as its Nash and coarse correlated equilibria.

Finally, we connect to recent works dealing with correlated and coarse correlated equilibria in mean-field games.
While well-known in game theory as generalizations of Nash equilibria (<cit.>), correlated and coarse correlated equilibria have been considered in the mean-field games literature very recently.
We refer to <cit.> and <cit.> for discrete-time finite-state mean-field games, and to <cit.> for continuous-time mean-field games.
The latter two papers are particularly relevant in our context, since we build on the definition and intuitions developed therein.





Structure of the paper.
The rest of the paper is organized as follows: Section <ref> presents the N-player game, and Section <ref> introduces the corresponding mean-field formulation.
Section <ref> details the assumptions and includes some auxiliary control-theoretic results.
In Section <ref>, the mean-field control problem is solved, and its connection with the central planner's optima are established.
Section <ref> characterizes both coarse correlated equilibria and Nash equilibria in the mean-field game, and discusses their relationships with the equilibria in the N-player game.
Section <ref> numerically illustrates the findings from previous sections. Finally, the sec:appendix contains technical proofs and lemmata.





§ THE N-PLAYER GAME


Let (Ω, ,𝔽 = (_t)_t ≥ 0^-,) be a filtered probability space satisfying the usual assumptions.
Let (W^i)_i ≥ 1, W be a sequence of independent 𝔽-Brownian motions, and let ξ, (ξ^i)_i ≥ 1 be a sequence of i.i.d. positive random variables with distribution μ_0 ∈𝒫(_+).
We assume that they are independent from W and (W^i)_i ≥ 1, and they are _0^--measurable.

We consider the following set of strategies, to be subject to further restrictions in the following:

    :=    {ν:Ω×_+ →_+,  t ↦ν_t 
       ν_0^- = 0 [ν_T]<∞ ∀  T > 0 }.


Let N ≥ 2.
We denote a vector of strategies (ν^1,…,ν^N) ∈^N by ν^N. We refer to ν^N∈^N as a strategy profile.
We denote by ν^-i,N=(ν^1,…,ν^i-1,ν^i+1,…,ν^N) the vector of strategies of players j ≠ i, and we denote the vector of strategies ν^N also by (ν^i,ν^-i,N).


Let δ, σ be in _+.
For any strategy profile ν∈^N, we consider the following dynamics:

    dX^ν^i_t=-δ X^ν^i_tdt+σ X^ν^i_t dW^i_t+dν^i_t,     X_0^-^ν^i=ξ^i,

for any i=1,…,N.
Observe that, for any ν^N ∈^N, there exists a unique strong solution to (<ref>) (see, e.g., <cit.>).
Actually, one has

    X^ν^i_t = X^i,0_t( ξ^i + ∫_0^t dν^i_s/X^i,0_s),

where X^0 = (X^1,0,…,X^N,0) denotes the uncontrolled solution of (<ref>), that is, the one associated to ν^i ≡ 0. 
Moreover, for any i=1,…,N, we define the flow of empirical averages of players j ≠ i by

    θ^N,ν^-i,N_t := 1/N-1∑_j  i X_t^ν^j,    t ≥ 0^-.


Let α, β in (0,1), q > 0.
Each player is associated with the following reward functional:

    ^N(ν^i,ν^-i,N) = lim inf_ T ↑∞1/T[ ∫_0^T (X^ν^i_t)^α( θ^N,ν^-i,N_t )^βdt - q ν^i_T],

which can possibly be infinite.
Occasionally, we use the notation π(x,θ) = x^αθ^β, for any (x,θ) ∈_+^2, and we write π_x(x,θ) = ∂_x π(x,θ) and analogously π_θ(x,θ) = ∂_θπ(x,θ).

When dealing with N-player games, we consider open-loop strategies. Roughly speaking, we allow each player to observe the noises of all players, as well as their initial position.
To this extent, denote by 𝔽^N = (^N_t)_t ≥ 0^- be the -augmentation of the filtration generated by the Brownian motions (W^i)_i=1^N and initial data (ξ^i)_i=1^N.

We say that a process ν∈ is an open-loop strategy for the N-player game if ν is 𝔽^N-progressively measurable. We denote the set of open-loop strategies for the N-player game by _N.


We are interested in different kinds of equilibria in the N-player system.
We deal both with the cooperative case and the competitive framework.



 §.§ Cooperative framework

In the competitive case, we look for Pareto efficient strategy profiles, according to the following definition:


Let 𝒞⊆_N^N.
A strategy profile ν̂∈𝒞 is Pareto efficient in the class 𝒞 if there does not exist any other ν∈𝒞 so that

    ^N(ν^j,ν^-j) ≥^N(ν̂^j,ν̂^-j),   ∀ j=1,…,N, 
       ^N(ν^i,ν^-i) > ^N(ν̂^i,ν̂^-i),   for some  i.


In other words, a strategy profile is Pareto efficient in 𝒞 if there does not exist any other strategy profile in 𝒞 which makes each player at least as well off and one player strictly better off.
To search for Pareto efficient strategy profiles, we associate to the dynamics (<ref>) and payoff functionals (<ref>) an N-dimensional control problem.
We consider the following functional

    ^N (ν) := 1/N∑_i=1^N ^N(ν^i,ν^-i),   ν∈𝒜_N^N,

which can be regarded as a welfare utility for the N-player system.


Let ≥ 0, 𝒞⊆^N_N.
A strategy profile ν̂ = (ν̂^1,…,ν̂^N) ∈𝒞 is -optimal for the central planner optimization problem within the set of strategy profiles 𝒞 if 

    ^N(ν̂) ≥^N(ν)-ε,  ∀ ν∈𝒞.

If = 0, we say that the strategy profile ν̂ is optimal for the central planner within the set of strategy profiles 𝒞.

When dealing with the central planner's optimization problem, the players are referred to as agents, since there is no competition between them: the central planner picks herself a strategy for each player in order to maximize the welfare utility functional ^N.
As a consequence, agents are not allowed to unilaterally deviate from the central planner's strategy profile.
It can be easily show that if a strategy profile is an optimum of the central planner maximization problem, it is Pareto efficient as well.






 §.§ Competitive framework

We consider the notion of coarse correlated equilibria in the N-player game, which allows for correlation between players' strategies, in the sense described below.
It comprehends the more common notion of Nash equilibria as the particular case in which players' strategies are not correlated.

We assume the following structural condition on the σ-algebra _0^-:
U
The σ-algebra _0^- is large enough to support a _0^--measurable uniform random variable independent of the initial data ξ, (ξ^i)_i ≥ 1 and the noises W, (W^i)_i ≥ 1.



Next, we introduce correlation between players' strategies.

A correlation device is any random variable Z: (Ω,,) → (,ℬ_) so that Z is _0^--measurable and independent of ξ, W, (ξ^i)_i≥ 1 and (W^i)_i ≥ 1.




We define a correlated strategy profile as a pair (Z,λ) so that

    
  * Z is a correlation device;
    
  * λ = (λ^i)_i=1^N is an admissible recommendation to the N players; that is, for each i=1,…,N, λ^i=(λ^i_t)_t ≥ 0^- belongs to  and it is progressively measurable with respect to the -augmentation of the filtration (σ(Z)∨^N_t)_t ≥ 0^-.



We now assign dynamics and rewards of each player. To do so, we must distinguish two cases: Suppose that each player i follows the recommendation λ^i. Then, players' state dynamics are given by (<ref>), and each player gets the reward ^N(λ^i,λ^-i), with ^N given by (<ref>).
Suppose player i deviates, while other players stick to the correlated strategy profile λ^-i.
The deviating player will pick instead an open loop strategy ν∈_N.
Her dynamics are given by (<ref>) and her reward is given by ^N(ν^i,λ^-i), with ^N given by (<ref>).


Let ε≥ 0, ℬ⊆_N.
A correlated strategy profile (Z,λ) is an ε-coarse correlated equilibrium (ε-CCE) of the ergodic N-player game within the set of strategies ℬ, if for any i=1,…, N, we have

    ^N(λ^i,λ^-i) ≥^N(ν,λ^-i)-ε,  ∀ ν∈ℬ.

If ε = 0, we say that the correlated strategy profile (Z,λ) is a coarse correlated equilibrium (CCE) of the ergodic N-player game within the set of strategies ℬ.


We give the following interpretation of correlation devices and the correlated strategy profiles: A correlation device or a mediator runs a lottery over open loop strategies.
The extraction of the strategy happens before the game starts and it is independent of the of the initial data and idiosyncratic shocks that determine the random evolution of players' states.
These features are captured by the fact that the random variable Z is _0^--measurable and it is independent of (ξ^i)_i ≥ 1 and (W^i)_i ≥ 1.
The correlation device Z introduces extra randomness in the game, but is not exogenous in the sense of a common noise (see, e.g., <cit.>): Indeed, it is picked by the moderator as part of her recommendation to the players.
Note that existence of the correlation device Z is guaranteed by Assumption <ref>.

We interpret deviations in the following way: Each player must decide whether to commit to moderator's lottery before the extraction happens, only by relying on the information given by the law of the correlated strategy (Z,λ), which is assumed to be common knowledge between the players.
If a player does not commit, she will pick a strategy without any information on the outcome of the extraction.
Notice that, since the deviating player has only knowledge of the law of the correlated strategy profile  (Z,λ), she will use a strategy ν∈_N, which is, in particular, independent of the correlation device Z; consequently, her state process is independent of Z.
We refer to <cit.> for more comments.



We observe that the definition of -coarse correlated equilibria for the N-player game extends the one of Nash equilibria, that we recall:

Let ε≥ 0, ℬ⊆_N.
A strategy profile ν^*=(ν^1,*,…,ν^N,*) ∈ℬ^N is an ε-Nash equilibrium (ε-NE) of the ergodic N-player game within the set of strategies ℬ, if for any i=1,…, N, we have

    ^N(ν^i,*,ν^-i,*) ≥^N(ν,ν^-i,*)-ε,  ∀ ν∈ℬ.

If ε = 0, we say that the strategy profile ν^* is a Nash equilibrium (NE) of the ergodic N-player game within the set of strategies ℬ.


Every -CCE (Z,λ) with deterministic correlation device Z is an -NE: It is enough to notice that, since Z is deterministic, the correlated strategy profile λ reduces to an open-loop strategy profile (ν^1,…,ν^N) in _N^N.
Conversely, any -NE induces an -CCE with deterministic correlation device.





§ THE ERGODIC MEAN-FIELD GAME

In order to determine -optimal solutions to the central planner problem and ε-equilibria in the competitive setting, we consider the mean-field counterparts of the optimization problem and game considered before.
We will then show in Sections <ref> and <ref> the relation between mean-field solutions to the N-player cooperative and competitive problems respectively.

We work on the same probability space (Ω,,𝔽,) defined in the previous section.
Given a strategy ν∈, we consider the following dynamics:

    dX^ν_t = -δ X^ν_tdt + σ X^ν_t dW_t +dν_t,    X_0^-=ξ.

For any _0^--measurable non-negative random variable θ, possibly degenerate, we consider the following payoff functional to be maximized:

    (ν,θ) = lim inf_T ↑∞1/T[ ∫_0^T (X^ν_t)^αθ^βdt -q ν_T ].


Let  𝔽^ξ,W = (^ξ,W_t)_t ≥ 0^- be the -augmentation of the filtration generated by ξ and W.
Analogously to Definition <ref>, we consider the following strategies:

A process ν∈ is an open-loop strategy if it is progressively measurable with respect to the filtration 𝔽^ξ,W.
We denote the set of open-loop strategies by _mf.





 §.§ Cooperative framework

We address the mean-field counterpart of the central planner's maximization problem, which is given by the mean-field control (MFC) maximization problem.
Roughly speaking, this problem consists in maximizing the reward (<ref>) under the additional constraint θ=lim_t →∞[X^ν_t] =: [X^ν_∞], for every control choice ν.

In order to properly define the reward, we need to restrict the class of admissible controls.

We say that a strategy ν is admissible for the mean-field control problem if ν∈_mf and the process (X^ν_t)_t ≥ 0^- admits a unique stationary distribution p^ν_∞∈𝒫(_+).
We denote the set of admissible strategies for the stationary MFC problem by _MFC.


For any ν∈_MFC, denote by [X^ν_∞] the first moment of the corresponding limit measure p^ν_∞.
The payoff functional associated to a strategy ν∈_MFC is given by

    (ν,[X^ν_∞]) = lim inf_T ↑∞1/T[ ∫_0^T (X^ν_t)^α ([X^ν_∞])^βdt - q ν_T ].



An admissible control ν̂∈_MFC is an optimal control for the mean-field control problem if 

    (ν̂,[X^ν̂_∞]) ≥(ν,[X^ν_∞]),   ∀ ν∈_MFC.



The study of the central planner's optimization problem and its relation with the N-agent system is the content of Section <ref>: We show in Theorem <ref> that it is possible to completely characterize solutions of the MFC problem.
Then, in Theorem <ref>, we use the solution of the MFC problem to build a sequence of approximate central planner's optima in the underlying N-agent system, with vanishing approximating error.






 §.§ Competitive framework

We consider the mean field analogues of CCEs and NEs in the N-player game.
To this extent, we give the following definition:

We define a correlated stationary strategy as a triple (Z,θ_∞,λ) so that the following holds:

    
  * Z is a correlation device;
    
  * θ_∞ is a σ(Z)-measurable non-negative random variable;
    
  * λ=(λ_t)_t ≥ 0^- belongs to  and it is progressively measurable with respect to the -augmentation of the filtration (σ(Z)∨_t^ξ,W)_t ≥ 0^-.


In the following, we will denote the law of θ_∞ by ρ∈𝒫 (_+).

Let (Z,θ_∞,λ) be a correlated stationary strategy.
We now assign dynamics and payoff functional.
We distinguish the following two cases:
If the representative player decides to trust the mediator and so to follow her recommendation λ, then the dynamics is given by (<ref>) with λ instead of ν and the payoff is given by (λ,θ_∞), with  defined by (<ref>).
If instead the representative player chooses to deviate, she uses a strategy ν∈, her dynamics is given by (<ref>), and she gets the reward (ν,θ_∞).
Observe that, when the representative player deviates, her strategy ν is 𝔽^ξ,W-progressively measurable and therefore independent of θ_∞, since she has no information on the outcome of moderator's lottery.
As in the N-player game, the deviating player can only use her knowledge of the law of the correlated stationary strategy (Z,λ,θ_∞), which is assumed to be publicly known.
Nevertheless, θ_∞ still appears in her payoff.



A correlated stationary triple (Z,λ,θ_∞) is a coarse correlated equilibrium (CCE) for the ergodic MFG if the following holds:

    
  *  (λ,θ_∞) ≥(ν,θ_∞) for any ν∈,
    
  *  The process X^λ admits a stationary distribution and it holds
    
    θ_∞ = ∫__+ x p_∞(dx,θ_∞),

    where p_∞ is the stochastic kernel so that μ_∞(dx,dθ)=p_∞(dx,θ)ρ(dθ) with ρ = ∘θ_∞^-1 and μ_∞ = lim_t →∞∘(X^λ_t,θ_∞)^-1 in the weak sense.


We will refer to CCEs for the ergodic MFG as mean-field CCEs as well.



Property <ref> in Definition <ref> is equivalent to 

    θ_∞∼ w-lim_t →∞[X^λ_t |θ_∞].




The consistency condition <ref> in Definition <ref> should be read in the following way: the mediator imagines what the stationary mean θ_∞ will be, before the game starts, and gives a recommendation to each player according to her idea.
Since θ_∞ is expected to be stochastic only as the result of the mediator's randomization, we request it to be measurable with respect to the correlation device Z that the moderator uses to generate both the recommendation λ and the random stationary mean θ_∞ itself.
If all players commit to the mediator's lottery for generating recommendations, then the long-time average should be consistent with what imagined by the mediator.

The notion of CCE for the ergodic MFG extends the notion of notion of Nash equilibrium for the ergodic MFG, that we borrow from <cit.>:
 
A pair (ν^*, θ^*) ∈_mf×ℝ_+ is said to be a Nash equilibrium of the ergodic MFG if 

    
  *  (ν^*, θ^* ) ≥(ν, θ^* ), for any ν∈_mf;
    
  *  The optimally controlled process X^ν^* admits a limiting distribution p^*_∞∈𝒫(_+) satisfying
    
    θ^* = ∫_ℝ_+ x p^*_∞(dx).

    

We will refer to NEs for the ergodic MFG as mean-field NEs as well.
We stress that, differently from mean-field CCEs, when looking for Nash equilibria, θ^* is assumed to be deterministic.

As in the N-player game, and actually by exactly the same reasoning, every CCE for the ergodic MFG (Z,θ_∞,λ) with deterministic correlation device Z is an Nash equilibrium for the ergodic MFG as well.
Conversely, any Nash equilibrium for the ergodic MFG (ν^*,θ^*) induces a mean-field CCE with deterministic correlation device.

The study of the existence of CCEs in the ergodic MFG and its relation with the N-player game is the content of Section <ref>: We identify specific classes of correlated stationary strategies for which it is possible to state a sufficient condition to be a CCE.
Then we use CCEs in those classes to build a sequence of approximate CCEs the underlying N-player system, with vanishing approximating error.
As a byproduct, we get the same approximation result for NEs as well.




§ ASSUMPTIONS AND PRELIMINARY RESULTS


On top of Assumption <ref>, we assume the following structural condition on the coefficients of the SDE:
D
The parameters δ and σ satisfy the following condition:

    2δ - σ^2 > 0.

Moreover, μ_0 admits a finite second moment. 



Let X^0 be the solution of (<ref>) when the policy ν is identically equal to 0.
Assumption <ref> is a dissipativity assumption on the square of X^0: indeed, by Itô's formula, we have

    (X^0_t)^2 = ξ^2 + ∫_0^t (-δ +1/2σ^2)(X^0_s)^2ds + ∫_0^tσ X^2_s dW_s,

which is then square-integrable and dissipative. Notice that the same assumption is assumed in <cit.>.


We state some important properties of the diffusion X^ν^a reflected upwards at some positive barrier a, which will be used through the whole manuscript.



  *  For any a > 0, let p_a ∈𝒫(_+) be given by

    p_a(dx) = 2δ + σ^2/2 a^2δ/σ^2 + 1x^-2δ/σ^2 -2_{ x ≥ a} dx.

For any 0 ≤ k ≤ 2, the measure p_a admits finite k-moment.
In particular, it holds

    ∫__+ x p_a(dx) = 2δ + σ^2/2δa.

Moreover, the map _+ ×ℬ__+∋ (a,B) ↦ p_a(B) defines a stochastic kernel from _+ to ℬ__+.


  *  For any a > 0, there exists a unique strong solution (X^ν^a_t,ν^a_t)_t ≥ 0^- of the Skorohod reflection problem at the barrier a, i.e. a pair of processes so that equation (<ref>) is satisfied for any t, ν_0^- = 0, t ↦ν_t is non-decreasing, and ∫_0^∞_{ X^ν^a_s > a }dν^a_s = 0, -a.s..
The process ν^a is given by

    ν^a_t = ∫_0^t X^0_s   d ( sup_0 ≤ u ≤ s( a-X^0_u/X^0_u)^+),   ν^a_0^- = 0.

Moreover, the process X^ν^a is positively recurrent with stationary measure given by p_a.


  *  There exists a positive constant c so that

    sup_t ≥ 0[(X^ν^a_t)^2]  ≤ c (1 + a^2),   sup_T > 0[( 1/Tν^a_T )^2]  ≤ c(1 + a^2).



The proof is postponed to the sec:appendix.
For later use, we introduce the real function C(a,p), for (a,p) ∈^2_+, given by 

    C(a,p) =     (2δ + σ^2)( p/2δ + σ^2(1-α) a^α - q/2a  ).

Let ν^a is the policy that reflects the process X^ν^a solution to (<ref>) upwards à la Skorohod at the level a > 0.
By <cit.>,
it holds

    C(a,p) = lim inf_T ↑∞1/T[ ∫_0^T p·(X^ν^a_t)^α dt - q ν^a_T ] = lim_T ↑∞1/T[ ∫_0^T p·(X^ν^a_t)^α dt - q ν^a_T ]


In the following, we will need to solve several ergodic optimization problems of singular controls.
The existence of a unique solution to such problems is ensured by following Lemma:

Let λ∈ so that q δ - λ > 0 and p > 0.
Define the following function

    g(x,p,λ) := x^α p + λ x,

and consider the reward functional

    (ν,p,λ) := lim inf_T ↑∞1/T[ ∫_0^T g(X^ν_t,p,λ) dt - q ν_T ],

where X^ν=(X^ν_t)_t ≥ 0^- evolves accordingly to (<ref>).
Then, there exists a unique optimal control ν^* ∈ so that

    (ν^*,p,λ) = sup_ν∈(ν,p,λ).

Moreover, the process ν^* reflects the state process upwards à la Skorohod at the barrier a^*(p,λ) given by 

    a^*(p,λ) = ( 2αδ/2δ + σ^2(1-α)p/qδ - λ)^1/1-α.


The proof is postponed to the sec:appendix.

With respect to a smaller class of strategies, we state a first order optimality condition for a control ν̂, inspired by <cit.> (see also <cit.>).
Although only the necessary part will be needed, for the sake of completeness, we also show that it is sufficient under additional assumptions.


Let p > 0, λ > q δ.
Let 1 ≤ q, q' ≤∞ be Young conjugates, and define the set _2q as the set of controls ν∈ so that

    sup_T > 01/T[ ∫_0^T | X^ν_t |^2q dt  ] < ∞.

Let ν̂∈_2q so that 

    sup_T > 01/T[∫_0^T | (X^ν̂_t)^α - 2|^q']<∞,

if q'<∞, and so that

    sup_T > 0inf{C ≥ 0:   (X^ν̂_t)^α - 2≤ C  for dt ⊗ d-a.e. (t,ω) ∈ [0,T] ×Ω}  < ∞,

if q' = ∞.

    
  *  Suppose that ν̂ is optimal within the set _2q for the control problem with dynamics (<ref>) and reward (<ref>). Then, for every ν∈_2q, it holds
    
    lim sup_T ↑∞1/T[ ∫_0^T  g_x(X^ν̂_t,p,λ)( X^ν̂_t - X^ν_t ) dt - q(ν̂_T - ν_T) ] ≥ 0.

    
  *  Suppose that either ν̂ satisfies (<ref>) and 
    
    lim inf_T ↑∞1/T[ ∫_0^T g(X^ν̂_t,p,λ) dt - q ν̂_T ] = lim_T ↑∞1/T[ ∫_0^T g(X^ν̂_t,p,λ) dt - q ν̂_T ],

    or that ν̂ satisfies
    
    lim inf_T ↑∞1/T[ ∫_0^T  g_x(X^ν̂_t,p,λ) ( X^ν̂_t - X^ν_t ) dt - q(ν̂_T - ν_T) ] ≥ 0.

    Then ν̂ is optimal within the set _2q.


The proof is postponed to the sec:appendix.


Let a>0 and let ν^a be the policy that reflects the process X^ν̂ upwards à la Skorohod at the barrier a.
Then, the control ν^a is so that ( (X^ν^a_t)^α - 2≤ c  ∀ t ≥ 0) = 1, for a constant c > 0, since the reflected process is so that X^ν^a_t ≥ a for any t, and X^ν̂ belongs to _2 by Lemma <ref>.
Therefore, (<ref>) is satisfied, and we take q=1 in (<ref>).
Assumption <ref> and Lemma <ref> imply that X^ν^a satisfies  (<ref>). 
By Lemma <ref>, (<ref>) is satisfied as well.




We can restate Lemma <ref> in terms of linear conditions involving optional projections.
Consider the probability measure  equivalent to  defined via the Radon-Nykodim derivative

    d/d|__t = e^σ W_t - σ^2/2t,    t ≥ 0.

It can be shown that, for every ν∈, the state process X^ν can be represented as

    X^ν_t = e^-δ t M_t (ξ + ν̅_t ),

where ν̅ = (ν̅_t)_t ≥ 0 is defined via the identity ν_t =∫_0^t e^-δ sM_s dν̅_s.
Denote by  the expectation with respect to .
By taking advantage of Fubini's theorem and optional projections as in <cit.>, we can restate necessary condition (<ref>) as

    lim sup_T ↑∞1/T[ ∫_0^T  ( [ ∫_s^T e^-δ t g_x(X^ν̂_t,p,λ)dt  |_s ] -q e^-δ s) d(ν̂̅̂_s - ν̅_s) ] ≥ 0,    ∀ν∈_2q,

and analogously holds for (<ref>) and (<ref>).
While first order conditions for optimality are well known for both finite and infinite time horizon singular control problems (see, e.g., <cit.> among others), to the best of our knowledge, this is the first time in the literature that first order conditions for singular control problems with ergodic reward functionals have been derived.






§ COOPERATIVE CASE: MEAN-FIELD SOLUTION AND APPROXIMATION


In this section, we tackle the mean-field solution of the central planner's optimization problem defined in Definition <ref>.



 §.§ Mean-field solution

The first result, Theorem <ref>, regards existence and uniqueness of optimal solutions of the MFC problem.
In order to compute the optimal control, we use a Lagrangian multiplier type approach, to take care of the constraint on the stationary first order moment.
We first restrict to strategies so that the corresponding stationary mean is equal to some prescribed level θ≥ 0, we compute the optimal strategy within this smaller constrained set and finally we optimize over all possible values of the stationary mean.
A somehow similar approach has been used in <cit.>, for a MFC problem of impulse control.


If α + β < 1, there exists a unique optimal control ν̂ for the MFC problem.
Moreover, the process upwards à la Skorohod ν̂ reflects the state process at the barrier â given by

    â = [ 2(α + β)/q(2δ + σ^2(1-α))( 2δ+σ^2/2δ)^β] ^1/1 - α -β,

and the corresponding stationary mean is given by 

    θ̂ = ( 2δ+σ^2/2δ)^1-α/1 - α -β[ 2(α + β)/q(2δ + σ^2(1-α))]^1/1-α -β.

If instead α + β > 1, the problem is ill-posed, in the sense that 

    sup_ν∈_MFC(ν,[X^ν_∞]) = +∞.

Finally, if α + β = 1 and 

    2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ+σ^2)^α  < qδ,

the null control ν≡ 0 is optimal; otherwise, the problem is ill-posed.


We note that

    sup_ν∈_MFC(ν,[X^ν_∞]) = sup_θ > 0sup_ν∈_MFC
    [X^ν_∞] = θ(ν,θ) = sup_θ > 0sup_ν∈_MFC
    [X^ν_∞] = θ(ν,θ) +λ(θ)([X^ν_∞] -θ),

where λ:_+ → is any real function of the mean θ.
Moreover, for ν∈_MFC so that [X^ν_∞] = θ, we rewrite the right-hand side term of (<ref>) using ergodicity:

    (ν,θ) +λ(θ)( [X^ν_∞] -θ) = (ν,θ) + lim_T ↑∞1/T∫_0^T[λ(θ) X^ν_t -λ(θ) θ]dt 
        = -λ(θ) θ + lim inf_T ↑∞1/T[ ∫_0^T (X^ν_t)^αθ^β + λ(θ) X^ν_tdt - qν_T ] = -λ(θ)θ + (ν,θ^β,λ(θ)),

where  is defined in (<ref>).
We split the problem in the following three steps:

    
  *  For fixed θ and λ, show that there exists a unique optimal control of barrier type which minimizes J(ν,θ^β,λ) over _mf.
    Denote by â(θ,λ) the optimal reflection barrier.
    
  *  Show that for any θ>0 there exists a real value λ(θ) so that θ=[X^ν^â(θ,λ(θ))_∞] and deduce that
    
    sup_ν∈_MFC
    [X^ν_∞] = θlim inf_T ↑∞1/T[ ∫_0^T (X^ν)^αθ^β + λ(θ) X^ν_tdt - qν_T ] = (ν^â(θ,λ(θ)),θ^β,λ(θ)).

    
  *  Perform the optimization over θ∈_+.


As for Step <ref>, we restrict to λ < q δ, so that, by applying Lemma <ref> with p = θ^β, the unique optimal control is the reflection policy at the level â(θ,λ)=a^*(θ^β,λ) given by (<ref>).


As for Step <ref>, we look for λ(θ) so that the stationary distribution satisfies [X^ν^â(θ,λ(θ))_∞] = θ and the condition q δ - λ(θ) > 0 holds.
In view of (<ref>), this is equivalent to imposing

    2δ + σ^2/2δ2δ + σ^2(1-α) / 2αδqδ -λ/θ^β^1/α - 1 = θ,    q δ - λ(θ) > 0

which are both satisfied by 

    λ(θ) = qδ - 2δ+σ^2/2δ^1- α2δα/2δ + σ^2(1-α)θ^α + β - 1.

Therefore, by choosing λ(θ) as the Lagrangian multiplier in (<ref>), we have that 

    sup_ν∈_MFC(ν,[X^ν_∞]) = sup_θ>0( -λ(θ) θ + sup_ν∈_MFC
    [X^ν_∞] = θ(ν;θ,λ(θ) ) ) 
        = sup_θ>0( -λ(θ) θ +(ν^â(θ,λ(θ));θ,λ(θ)) ) = sup_θ>0(ν^â(θ,λ(θ)),θ) = sup_θ>0(ν^â(θ,λ(θ)),[X^ν^â(θ,λ(θ))_∞]).


We are left with performing the optimization over θ∈_+.
By exploiting (<ref>) and (<ref>), for every θ>0 we have 

    (ν^â(θ,λ(θ)),[X^ν^â(θ,λ(θ))_∞]) = C(â(θ,λ(θ)),θ)=2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ+σ^2)^αθ^α+β - qδθ.

Set

    f(θ) := 2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ+σ^2)^αθ^α+β - qδθ.

If α + β < 1, we have

    f'(θ) = (α + β ) 2δ/ 2δ+σ^2 ^α2δ + σ^2/2δ + σ^2(1-α)θ^α + β -1 -  qδ,

so that f”(θ) < 0 for every θ>0, i.e. f is strictly concave in _+.
This implies that there exists a unique maximizer θ̂.
By imposing f'(θ) = 0, we find the expression of θ̂ given by (<ref>), and by (<ref>) we find the expression of â in (<ref>).

If α + β >1, the function f defined in (<ref>) is unbounded, and therefore the MFC problem does not admit a maximizer.
Finally, suppose α + β = 1.
Then, the function f is just given by 

    f(θ) = ( 2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ+σ^2)^α  - qδ) θ.

Since f is linear in θ, we either have sup_θ>0f(θ) equal to 0 or +∞ depending on the sign of the coefficient.



Assumptions <ref> and <ref> are not needed to prove Proposition <ref>.
On the other hand, according to the previous result, restrictions on the parameters are needed in order to have existence of the optimal control.
Notice that, if an optimal control exists, it is always unique, by strict concavity of the reward functional.


For the sake of completeness and for later use, we derive the relationship between the Lagrangian multiplier λ(θ) and the constraint parameter θ.


Let ν = ν^a(θ) be the strategy which reflect the process X^ν^a(θ) upwards at the barrier a(θ) = 2δ/2δ + σ^2θ.
Let λ(θ) be given by (<ref>) and f given by (<ref>).
Then, for any θ > 0, it holds

    f'(θ) + λ(θ) = lim inf_T ↑∞1/T[∫_0^T π_θ(X^ν^a(θ)_t,θ)dt ].


The proof is postponed to the sec:appendix.


The same calculations of Lemma <ref> show that the following relation holds:

    λ(θ̂) =  ∂_θ(ν̂,θ̂) = ∂_θ( sup_ν∈_MFC(ν,[X^ν_∞]) ).

In this sense, the Lagrange multiplier λ(θ̂) can be regarded as the derivative of the value function with respect to the measure argument. 




 §.§ Approximation

We show that any solution to the MFC problem as given by Theorem <ref> induces a sequence (ν̂^N)_N ≥ 1 of approximate optimal strategy profiles for the central planner, with vanishing error.
Our approach is mainly inspired by <cit.>, although it uses different techniques due to the nature of our dynamics and payoff.


Let N ≥ 2.
We consider the following set 𝒞⊆_N^N of strategies for the central planner.

A strategy profile β^N = (β^1,N,…,β^N,N) is an admissible strategy profile for the  central planner optimization problem if β^i,N∈_N for any i=1,…,N, if they are exchangeable, in the sense that the triples (ξ^i,β^i,N,W^i)_i=1^N are exchangeable random elements, and ergodic, in the sense that the resulting N-dimensional process (X_t^β^1,N,…, X^β^N,N_t)_t ≥ 0 is ergodic, where X^β^i,N is given by (<ref>) for every i=1,…,N.
Moreover, every strategy β^j is such that

    sup_T > 01/T[ ∫_0^T | X^β^j_t |^2 dt  ] ≤ c,   ∀  j=1,…,N.

We denote the set of strategy profiles for the central planner by ^cp,c_N. 

Observe that the inclusion ^cp,c_N ⊆^N_N holds strictly.
The exchangeability assumption is well known in the MFC literature, when dealing with approximation results. We refer to <cit.> for more comments.
Observe that, in particular, exchangeability implies

    ^N (β^N) = ^N(β^i,N,β^-i,N),

for every i=1,…,N, for any strategy profile β^(N)∈^cp,c_N.


Let ν̂^N = (ν̂^1,…,ν̂^N) be the strategy profile that reflects each process X^ν̂^i upwards à la Skorohod at the level â given by (<ref>).
It holds

    lim_N →∞sup_β^(N)∈^cp,c_N^N(β^(N)) = lim_N →∞^N(ν^(N))= (ν̂,[X^ν̂_∞]).




Notice that, for every N ≥ 2, ν^N belongs to ^cp_N: since the sequence (ν̂^i,X^ν̂^i)_i ≥ 1 is i.i.d. as (ν̂,X^ν̂), the system is excheangeble.
Moreover, by <cit.>, the N-dimensional process (X^ν̂^i)_i = 1^N is a positively recurrent regular diffusion with ergodic measure ⊗_i = 1^N p̂_∞(dx_i), where we set p̂_∞ = p_â, with p_a given by (<ref>).
Moreover, up to choosing c large enough, the processes (X^ν̂^i)_i = 1^N satisfy (<ref>) by point <ref> of Lemma <ref>.

We show that

    lim_N →∞inf_β^(N)∈^cp,c_N ( ^MFC(ν̂) - ^N(β^(N))) ≥ 0,       lim_N →∞( ^MFC(ν̂) - ^N(ν^(N))) = 0.

Observe that, by Lemma <ref>, the inferior limit in the definition of ^MFC(ν̂^i) is actually a limit.
Then, by using the inequalities lim sup_n z_n -lim inf_n x_n ≥lim sup_n(z_n - y_n) and lim sup_n(z_n + y_n) ≥lim sup_n z_n + lim inf_n (y_n) and concavity of π(x,θ) = x^αθ^β jointly in (x,θ), it holds

    ^MFC (ν̂^i) - ^N(β^i,N,β^-i,N) ≥lim sup_T ↑∞1/T[ ∫_0^T ( π(X^ν̂^i_t,θ̂) - π(X^β^i,N_t,θ^N,β^-i,N_t) )dt - q(ν̂^i_T - β^i,N_T) ] 
       ≥lim sup_T ↑∞1/T[ ∫_0^T  ( π_x(X^ν̂^i_t,θ̂) + λ(θ̂))( X^ν̂^i_t - X^β^i,N_t ) dt - q(ν̂^i_T - β^i,N_T) ]  
         + lim inf_T ↑∞1/T[ ∫_0^T  ( -λ(θ̂)( X^ν̂^i_t - X^β^i,N_t ) + π_θ(X^ν̂^i_t,θ̂)(θ̂ - θ^N,β^-i,N_t) ) dt ] 
       ≥lim inf_T ↑∞1/T[ ∫_0^T  -λ(θ̂)( X^ν̂^i_t - X^β^i,N_t ) dt ]  + lim inf_T ↑∞1/T[∫_0^T π_θ(X^ν̂^i_t,θ̂)(θ̂ - θ^N,β^-i,N_t) dt ]

where we added and subtracted λ(θ̂)(X^ν̂^i_t - X^β^i_t) inside the time integral.
Last inequality follows from sublinearity of the lim inf and from Lemma <ref>, as the pair (X^ν̂^i,ν̂^i) has the same distribution as (X^ν̂,ν̂).
Moreover, since (X^ν̂^i)_i ≥ 1 are i.i.d. copies of X^ν̂, by Lemma <ref>, it holds

    λ(θ̂) = lim_T ↑∞1/T[∫_0^T π_θ(X^ν̂^i_t,θ̂)dt ]   ∀ i ≥ 1.

Since the second lim inf in the last line of (<ref>) is actually a limit, by using ergodicity and the identical distribution of (X^ν̂^i)_i=1^N and (X^β^i)_i=1^N respectively, we deduce

    ^MFC (ν̂^i) - ^N(β^i,N,β^-i,N) ≥lim inf_T ↑∞1/T[ ∫_0^T  ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) ( θ̂ - θ^N,β^-i,N_t ) dt ] 
        = N/N-1lim inf_T ↑∞1/T[ ∫_0^T  ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) ( 1/N∑_j=1^N (θ̂-X^β^j,N_t) - 1/N(θ̂-X^β^i,N_t) ) dt ],

where we used the identity 1/N-1∑_j ≠ 1 y_j = N/N-1(1/N∑_i = 1^N y_j - 1/Ny_i).
By taking the average over N, we have

    ^MFC (ν̂) - ^N(β^(N)) ≥N/N-1lim inf_T ↑∞1/T∫_0^T [ ( 1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) )   ) ( 1/N∑_j=1^N (θ̂-X^β^j,N_t)  ) 
          - 1/N^2∑_i=1^N (π_θ(X^ν̂^i_t,θ̂) -λ(θ̂))( θ̂-X^β^i,N_t ) ]dt 
       ≥ - c( lim sup_T ↑∞1/T∫_0^T [ |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) )   ||1/N∑_j=1^N (θ̂-X^β^j,N_t)  |]dt 
          + 1/Nlim sup_T ↑∞1/T∫_0^T [ |1/N∑_i=1^N (π_θ(X^ν̂^i_t,θ̂) -λ(θ̂))( θ̂-X^β^i,N_t ) |]dt ).

We study separately the two lim sup.
As for the the second one, by taking advantage the exchangeability of (X^ν̂^i)_i ≥ 1 and (X^β^i,N)_i=1^N, we get

    1/Nlim sup_T ↑∞1/T∫_0^T [ |1/N∑_i=1^N (π_θ(X^ν̂^i_t,θ̂) -λ(θ̂))( θ̂-X^β^i,N_t ) |]dt 
       ≤1/Nlim sup_T ↑∞( 1/T∫_0^T [ |π_θ(X^ν̂_t,θ̂) -λ(θ̂) |^2 ] dt )^1/2( 1/T∫_0^T [ |θ̂ - X^β^1,N_t |^2 ] dt )^1/2
       ≤c/Nlim sup_T ↑∞( 1/T∫_0^T [ |π_θ(X^ν̂_t,θ̂) -λ(θ̂) |^2 ] dt )^1/2,

that goes to 0 as N →∞, by definition of the set of strategy profiles ^cp,c_N.
As for the first term, by analogous computations, we get

    lim sup_T ↑∞1/T∫_0^T [ |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) )   ||1/N∑_j=1^N (θ̂-X^β^j,N)  |]dt 
       ≤lim sup_T ↑∞( 1/T∫_0^T [ |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2  ]dt )^1/2( 1/T∫_0^T [ |1/N∑_j=1^N (θ̂-X^β^j,N)  |^2 ]dt)^1/2
       ≤ c lim sup_T ↑∞( 1/T∫_0^T [ |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2  ]dt )^1/2,

for a constant c independent of N, by using the bound (<ref>) and exchangeability of β^(N)∈^cp,c_N.
By the ergodic ratio theorem, it holds 

    lim_T ↑∞1/T∫_0^T  |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2  dt = ∫__+^N|1/N∑_i=1^N ( π_θ(x_i,θ̂) -λ(θ̂)  ) |^2 ⊗_i=1^N p̂_∞(dx_i),

-a.s.. We show that the right hand-side is uniformly integrable.
Take r = 1α > 1.
By Jensen inequality and identical distribution of (X^ν̂^i)_i ≥ 1, we have

    [( 1/T∫_0^T  |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2 dt )^r ] ≤C/T[ ∫_0^T  |( 1/N∑_i=1^N π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2r dt ] 
       ≤ C ( |λ(θ̂)|^2r + 1/N∑_i=1^N  1/T∫_0^T  [ |π_θ(X^ν̂^i_t,θ̂) |^2r] dt  )  ≤  C ( 1 + 1/T∫_0^T  [ |π_θ(X^ν̂_t,θ̂) |^2r] dt )  
       ≤ C ( 1 + 1/T∫_0^T [| (X^ν̂_t)^α|^2r] dt ) = C ( 1 + 1/T∫_0^T [  | (X^ν̂_t) |^2] dt ) ≤ C(1 + â^2),

where we used Lemma <ref> in the last estimate.
Since last estimate holds for any T>0 and r>1, we deduce that the right hand-side of (<ref>) is uniformly integrable.
By, e.g., <cit.>) this yields

    lim_T ↑∞1/T[ ∫_0^T  |1/N∑_i=1^N ( π_θ(X^ν̂^i_t,θ̂) -λ(θ̂) ) |^2 dt ] = ∫__+^N|1/N∑_i=1^N ( π_θ(x_i,θ̂) -λ(θ̂)  ) |^2 ⊗_i=1^N p̂_∞(dx_i).

We conclude by invoking the law of large numbers:
let (X_i)_i ≥ 1 be a sequence of i.i.d. random variables with law p̂_∞(dx).
Therefore,  by Lemmata <ref> and <ref>, the sequence (Y_i)_i ≥ 1 defined by Y_i = π_θ(X_i,θ̂) -λ(θ̂) is i.i.d., square integrable and centered, which yields

    ∫__+^N-1|1/N∑_i=1^N ( π_θ(x_i,θ̂) -λ(θ̂) ) |^2 ⊗_i= 1^N p̂_∞(dx_i) = [ |1/N∑_i = 1^N Y_i |^2 ] = [| Y_1] | ^2/N→ 0.


Finally, lim_N →∞^N(ν^(N)) = ^MFC(ν̂) follows form ergodicity of the processes (X^ν̂^i)_i=1^N and the law of large numbers, analogously as before.






§ COMPETITIVE CASE: MEAN-FIELD EQUILIBRIA AND APPROXIMATION

In this section, we focus on coarse correlated equilibria for the MFG, as defined by Definition <ref>.
Since the set of CCEs is typically very wide and it is difficult to characterize in a continuous time setting, following the procedure outlined in <cit.>, we restrict our analysis to specific classes of correlated stationary strategies for which we are able to state a sufficient condition for being a CCE for the ergodic MFG.
With respect to <cit.>, we move a step forward, and show that every CCE in these classes induces a sequence of approximate CCEs in the underlying N-player game with vanishing error.
More specifically, we establish the following:

    
  * We fix a correlated stationary strategy (Z,λ,θ_∞).
    We suppose that the representative player decides to ignore the moderator's recommendation, and compute her best deviating strategy, i.e.
    
    U^* = argmax_ν∈_mf(ν,θ_∞).

    This is the content of Proposition <ref>.
    
    
  * We define specific classes of correlated flows (Z,λ,θ_∞) so that the consistency condition <ref> is satisfied.
    This is established in Propositions <ref> and <ref>.
    
    
  * For (Z,λ,θ_∞) in such classes, we express the optimality condition <ref> as an inequality involving the law of θ_∞ only, thus deriving a sufficient condition for the existence of CCEs. This is the content of Propositions <ref> and <ref>.

    
  * We show that every mean-field CCEs in each class induces a sequence of aprroximate CCEs in the underlying N-player game with vanishing error.
    This is the content of Theorems <ref> and <ref>.


We consider two classes of correlated stationary strategies: while in both classes the correlating device is the random mean θ_∞ itself, in the first class the recommendation λ^r is a σ(θ_∞)-measurable regular control, while in the second one, the recommendation λ^s is a policy of reflection type at a random barrier a(θ_∞).
Surprisingly, the sufficient condition of the two classes differ only by a constant.
Moreover, in Theorem <ref>, we explicitly characterize existence and uniqueness of Nash equilibria. We find that they belong to class of CCEs with recommendation of reflection type, so that the same approximation result applies.





 §.§ The Deviating Player Problem

Suppose that the representative player decides to ignore the moderator's recommendation.
By definition of CCE for the ergodic MFG, the deviating player must choose her strategy ν∈_mf only by knowing the joint law of the correlated stationary strategy (Z,λ,θ_∞), which is assumed to be publicly known, and not by observing its realizations.
Since ν∈_mf, it follows that X^ν is 𝔽^ξ,W-adapted as well and thus independent of the random variable θ_∞, which implies that deviating player's payoff can be written as:

    (ν,θ_∞) = lim inf_T ↑∞1/T( ∫_0^T [ [ (X^ν_t)^αθ_∞^β|^ξ,W_t ] ] - q[ [ν_T |^ξ,W_T] ] ) 
        = lim inf_T ↑∞1/T[ ∫_0^T (X^ν_t)^α[θ_∞^β]dt - q ν_T ].

Observe that deviating player's payoff functional depends on θ_∞ only through its expectation.


There exists a unique optimal strategy for the deviating player U^* ∈_mf which reflects the process X^U^* upwards á la Skorohod at the level a^*, where a^* is given by

    a^* = ( 2α/q(2δ + σ^2(1 - α)))^1/1-α([θ_∞^β])^1/1- α.



Since the payoff functional of the deviating player is given by (<ref>), it is enough to apply Lemma <ref> with λ = 0 and p = [θ_∞^β].


In the following, we set

    K :=  2α/q(2δ + σ^2(1 - α))^1/1-α,

so that the optimal policy of the deviating player is the reflection at the level a^* = K ([θ_∞^β])^11-α.




 §.§ Regular recommendation

Let 𝒢^r be the set of correlated stationary strategies (Z,λ^r,θ_∞) so that θ_∞∈ L^2(_0^-) independent of ξ and W, Z=θ_∞ and 

    dλ^r_t = δθ_∞ dt,    t ≥ 0,

so that, in particular, the recommended strategy λ^r is a σ(θ_∞)-measurable regular control.


Let (Z,θ_∞,λ^r) ∈𝒢^r.
Define

    m'_∞,θ(x) = 2/σ^2exp2δθ/σ^2x^-2δ/σ^2 - 2exp-2δ/σ^2θ/x,

and let p^r_∞(dx,θ) be the stochastic kernel from _+ to ℬ__+ defined by

    p^r_∞(dx,θ)= x^ -2δ/σ^2  - 2exp -2δ/σ^2θ/x/∫_0^∞ y^ -2δ/σ^2 -2  exp -2δ/σ^2θ/y dy  dx.

Then, the triple (Z,θ_∞,λ^r) is a correlated stationary strategy so that the consistency condition (<ref>) is satisfied.
In particular, it holds μ^r_∞(dx,dθ) = lim_t →∞∘(X^λ^r_t,θ_∞)^-1 = p^r_∞(dx,θ)ρ(dθ).


The measurability requirements on the triple (Z,θ_∞,λ^r) are clearly satisfied.
Let X^λ^r the state process controlled by λ^r, which satisfies 

    dX^λ^r_t = δ(θ_∞ - X^λ^r_t)dt + σ X^λ^r_t dW_t,    X^λ^r_0 = ξ.

We show that the joint law of (X^λ^r_t,θ_∞) converges weakly to μ^r_∞ as t →∞.
To this extent, it is enough to verify that the regular conditional probability of X^λ^r_t with respect to θ_∞ = θ, which we denote by p^r_t(dx,θ), converges weakly to p^r_∞(dx,θ) as t →∞, for ρ-a.e. θ∈_+.
Conditionally to θ_∞ = θ, X^λ^r_t satisfies the following equation:

    dX^λ^r,θ_t = δ(θ - X^λ^r,θ_t )dt + σ X^λ^r,θ_t dW_t,    X^λ^r,θ_0 = ξ.

By Lemma <ref>, we have ∫_0^∞ m'_∞,θ(x)dx < ∞, which implies that the diffusion X^λ^r,θ is positively recurrent for every θ > 0.
Thus, the measure p^r_∞(dx,θ) is the unique stationary distribution and p^r_t(dx,θ) → p^r_∞(dx,θ) in total variation norm (see, e.g., <cit.>).
As for equality (<ref>), define φ=(φ_t(θ))_t ≥ 0, θ>0 as

    φ_t(θ) = e^-δ t[ξ] +θ(1 - e^δ t).

By Itô's formula, it follows that φ_t(θ_∞) = [X^λ^r_t |θ_∞] for every t≥ 0, -a.s., which implies that θ_∞= lim_t →∞[X_t |θ_∞] -a.s., and therefore condition (<ref>) is satisfied, and so (<ref>).




A correlated stationary strategy (Z,θ_∞,λ^r) in the class 𝒢^r
is a mean-field CCE if and only if the following inequality is satisfied:

    c_β ([θ_∞^β])^1/ 1-α + c_1 [θ_∞] ≤  c_α+β[θ_∞^α+β],

where c_β, c_α+β and c_1 are positive constants defined by

    c_β := (2δ+ σ^2)q/22α/q(2δ + σ^2(1 - α))^1/1-α 1 -α/α,         c_1 := δ q,           c_α+β := 2δ/σ^2^αΓ( 2δ/σ^2 + 1 - α)/Γ( 2δ/σ^2 + 1 ) .



By Proposition <ref>, (Z,θ_∞,λ^r) satisfies the consistency condition (<ref>).
Let U^* be the optimal control for the deviating player given by Proposition <ref>.
Since (U^*,θ_∞) = max_ν∈_mf(ν,θ_∞), we just need to verify that the inequality (λ^r,θ_∞) ≥(U^*,θ_∞) is equivalent to (<ref>).
Since U^* is a reflection policy at the level a^* given by (<ref>), formulae (<ref>) and (<ref>) yield

    (U^*,θ_∞) = C(a^*,[θ_∞^β]) 
        = 2δ + σ^2 /2δ + σ^2(1 - α)[θ_∞^β]( K([θ_∞^β])^1/1-α)^α - q 2δ + σ^2/2 K([θ_∞^β])^1/1-α
        = (2δ+ σ^2)1/2δ + σ^2(1-α)K^α - q1/2K ([θ_∞^β])^1/1-α = c_β([θ_∞^β])^1/1-α,

by noticing that, with K given by  (<ref>), it holds

    (2δ+ σ^2)1/2δ + σ^2(1-α)K^α -q/2K  = 2δ+ σ^2/2 q K1 -α/α = c_β.


As for the payoff associated to the representative player, conditionally to θ_∞ = θ and exploiting ergodicity, it holds

    lim_T ↑∞   1/T[ ∫_0^T (X^λ^r_t)^αθ^β dt -qλ^r_T|θ_∞ = θ]  = ∫_0^∞θ^β x^α p^r_∞(dx,θ)- δ q θ
        = 2δ/σ^2^αΓ( 2δ/σ^2 - α  + 1)/Γ( 2δ/σ^2 + 1 )θ^α +β - δ q θ ,

where last equality follows from the definition of p^r_∞(dx,θ) and Lemma <ref>.
Moreover, by (<ref>), we have the bound

    1/T   [ ∫_0^T (X^λ^r_t)^αθ^β dt -qλ^r_T|θ_∞] ≤θ^β/T[∫_0^T(1 + X^λ^r_t)dt |θ_∞] + δ q θ_∞
       ≤θ^β(1 + sup_t ≥ 0[X^λ^r_t|θ_∞] )+ δ q θ_∞≤ C(1 + θ_∞^2),

which is integrable by assumption.
Therefore, by dominated convergence theorem, we can exchange limit and expectation to conclude

    (λ^r,θ_∞) = lim_T ↑∞1/T[ [ ∫_0^T (X^λ^r_t)^αθ^β dt -qλ^r_T|θ_∞] ] = [ lim_T ↑∞1/T[ ∫_0^T (X^λ^r_t)^αθ^β dt -qλ^r_T|θ_∞] ] 
        = 2δ/σ^2^αΓ( 2δ/σ^2 - α  + 1)/Γ( 2δ/σ^2 + 1 )∫_0^∞θ^α +βρ(dθ) - δ q ∫_0^∞θρ(dθ),

By comparing this equation with equation (<ref>), we get equation (<ref>).




Observe that, if α + β≤ 1, for this result to hold true it is enough to require θ_∞ to be in L^1; Assumption <ref> is not needed as well.


Finally, we show how to use a CCE in the class 𝒢^r to build a sequence λ^N of _N-CCE in the N-player game with _N → 0 as N →∞.
To this extent, we consider the following set of strategies ℬ⊆_N:

Let c > 0. A strategy ν be in _N is c-admissible if 

    lim sup_T ↑∞1/T[∫_0^T | X^ν_t |^2 dt ] ≤ c.

We denote by _N,c the set of c-admissible strategies for the N-player game.


Starting from a correlated stationary strategy (Z,θ_∞,λ^r) in the class 𝒢^r, we define the following correlated strategy profiles for the N-player game: take Z = θ_∞ as correlation device, and set, for any i ≥ 1,

    dλ^i,r_t = δθ_∞ dt.

Then, when player i plays accordingly to moderator's suggestion, her dynamics are hold by the following equation:

    dX^i,r_t = δ(θ_∞ - X^i,r_t)dt + σ X^i,r_t dW^i_t,    X^i,r_0 = ξ^i.

Observe that, for each i ≥ 1, the triple (X^i,r,λ^i,r,θ_∞) has the same law as (X,λ,θ_∞).
Moreover, while not independent, the processes (X^i,r)_i ≥ 1 are conditionally independent given θ_∞.


Let (Z,θ_∞,λ^r) be a CCE in the class 𝒢^r.
Let λ^N = (λ^i,r)_i=1^N, with λ^i,r defined by (<ref>).
Then, for any c > 0, the correlated strategy profile (θ_∞, λ^N) defines an ε_N-CCE for the N-player game within the set of strategies _N,c, with ε_N → 0 as N→∞.



For every N ≥ 2, set 

    ε_N := sup_ν∈_N,c( _N(ν,λ^-i,N) - _N(λ^i,r,λ^-i,N) ).

Notice that, by symmetry, ε_N is independent of i=1,…,N.
Clearly (θ_∞,λ^N) is an _N-CCE for the N-player game within the set of strategies _N,c.
We show that ε_N vanishes as N goes to ∞.
Note that, for any ν in _N,c, we have

    _N(ν,λ^-i,N) -    _N(λ^i,r,λ^-i,N) = ( _N(ν,λ^-i,N)  - (ν,θ_∞) ) 
        + ( (ν,θ_∞) - (λ^i,r,θ_∞) ) + ( (λ^i,r,θ_∞) - _N(λ^i,r,λ^-i,N) )

where  is defined by (<ref>).
We treat separately each of the three terms in the right-hand side of (<ref>).

As for the first term, by Cauchy-Schwartz inequality and using the inequality lim inf_n →∞α_n - lim inf_n →∞β_n≤lim sup_n →∞(α_n - β_n), we have the following estimates:

    |    (ν,θ_∞) - _N(ν,λ^-i,N) |
       ≤lim sup_ T ↑∞( 1/T∫_0^T [ (X^i,ν_t)^2α]dt )^1/2(1/T∫_0^T [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 ]dt )^1/2
       ≤( 1 + lim sup_ T ↑∞1/T∫_0^T [ | X^i,ν_t |^2 ]dt )^1/2( lim sup_ T ↑∞1/T∫_0^T [ [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] ]dt )^1/2
       ≤   (1+c)^1/2( [ lim sup_ T ↑∞1/T∫_0^T  [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] dt ] )^1/2,

where in the last inequality we exchanged limsup and expectation by reverse Fatou's lemma with the integrable upper bound C(1 + θ_∞^2).
Indeed, by recalling that (X^j)_j ≠ i are i.i.d. as X conditionally to θ_∞, we have the following estimates

    [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] ≤ C( 1 + θ_∞^2 + 1/N-1∑_j ≠ i[ | X^j_t |^2|θ_∞] ) ≤ C(1 + θ_∞^2),

where last inequality holds thanks to Lemma <ref>.
By Lemma <ref>, we then have

    lim sup_ T ↑∞   1/T∫_0^T [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] dt 
        = ∫__+^N-1|( 1/N-1∑_j ≠ ix_j )^β - θ_∞^β|^2 ⊗_j ≠ ip^r_∞(dx_j,θ_∞),   -a.s.

which converges to 0 in expectation as N goes to infinity by Lemma <ref> and dominated convergence theorem, with c(1+θ_∞^2) as the integrable upper bound.

As for the second term, we claim that (ν,θ_∞) - (λ^i,r,θ_∞) ≤ 0 for any ν∈_N,c.
Indeed, observe that, since ν∈_N is independent of θ_∞ by definition of admissible strategies for the N-player game, the proof of Proposition <ref> shows that 

    sup_ν∈_N(ν,θ_∞) = (U^i,*,θ_∞),

where U^i,* is the policy that reflects the process X^i,U^i,* upward à la Skorohod at the level a^* given by (<ref>).
In particular, X^i,U^i,* as the same distribution of the process X^U^*.
Therefore, we have

    sup_ν∈_N,c( (ν,θ_∞) - (λ^i,r,θ_∞) ) ≤sup_ν∈_N(ν,θ_∞) - (λ,θ_∞) ≤(U^*,θ_∞) - (λ,θ_∞) ≤ 0

where we used the inclusion _N,c⊆_N, the fact that (X^i,r,λ^i,r,θ_∞) has the same distribution as (X,λ,θ_∞) and the optimality property <ref> of CCE of the ergodic MFG.

As for the third term, taking advantage of the conditional independence and identical distribution of (X^λ^i,r)_i ≥ 1 and by analogous estimates as in (<ref>), we have

    |    (λ^i,r,θ_∞) - _N(λ^i,r,λ^-i,N) |
       ≤ C (1 + [θ_∞^2])^1/2( [ lim sup_ T ↑∞1/T∫_0^T [ |( θ^N,λ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] dt ])^1/2

and we conclude the proof by applying Lemmata <ref> and <ref> as before.



It is worth to notice that the integrability condition (<ref>) that defines c-admissible strategies can be weakened  at the price of more integrability requirements on θ_∞, ξ and the diffusion X^0.
Indeed, let q=β1-α, k=1+⌈ q ⌉, and suppose that 2δ-(k-1)σ>0, [ξ^k]<∞ and [θ_∞^k]<∞, which imply that the estimates in point <ref> of Lemma <ref> holds up to the k-th moment.
Then, up to little modification of the proof, one could consider strategies ν∈_N so that

    lim sup_T ↑∞1/T[∫_0^T X^ν_s ds] ≤ c.
 






 §.§ Singular Recommendation

We now look for a policy λ^s of reflection type at a random barrier a(θ_∞).
Let 𝒢^s be the set of correlated stationary strategies (Z,λ^s,θ_∞) so that Z=θ_∞, θ_∞∈ L^2(_0^-) independent of ξ and W, and λ^s is the control that reflects the process X^λ^s upwards à la Skorohod at the random level

    a(θ_∞) = 2δ/2δ + σ^2θ_∞.




Let (Z,λ^s,θ_∞) in 𝒢^s.
Let p^s_∞(dx,θ) be the stochastic kernel from _+ to ℬ__+ defined by

    p^s_∞(dx,θ)= p_a(θ)(dx),

where p_a is the family of measures defined by (<ref>).
Then, the triple (Z,θ_∞,λ^s) is a correlated stationary strategy so that the consistency condition (<ref>) is satisfied.
In particular, it holds μ^s_∞(dx,dθ) = lim_t →∞∘(X^λ^s_t,θ_∞)^-1 = p^s_∞(dx,θ)ρ(dθ).


Since the map _+ ×ℬ__+∋ (a,B) ↦ p_a(B) defines a stochastic kernel from _+ to ℬ__+, the kernel (<ref>) is well defined.
As in the proof of Proposition <ref>, we show that the joint law of X^λ^s_t and θ_∞ converges weakly to μ^s_∞ as t →∞.
Indeed, conditionally to θ_∞ = θ, X^λ^s_t satisfies the equation (<ref>) with ν replaced by λ^s,θ, where λ^s,θ reflects the process X^λ^s,θ upwards at the level a(θ), for ρ-a.e. θ∈_+.
By Lemma <ref>, the reflected process X^λ^s,θ admits p_a(θ) given by (<ref>) as the unique invariant distribution.
This implies that the regular conditional probability of X^λ^s_t with respect to θ_∞, that we denote by p^s_t(dx,θ), converges weakly to p^s_∞(dx,θ) as t →∞ for ρ-a.e. θ > 0.
Consistency condition (<ref>) follows from the definition of a(θ_∞) and (<ref>).



A correlated stationary strategy (Z,θ_∞,λ^s) in 𝒢^s is a mean-field CCE for the ergodic MFG if and only if the following inequality is satisfied:

    c_β ([θ_∞^β])^1/1-α + c_1[θ_∞] ≤c_α+β[θ_∞^α+β],

where c_1 and c_β are given by (<ref>) and c_α+β is given by 

    c_α+β := 2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ + σ^2)^α.



As in the proof of Proposition <ref>, it is enough to verify that the inequality (λ^s,θ_∞) ≥(U^*,θ_∞) is equivalent to (<ref>).
By (<ref>), the payoff of the deviating player is equal to

    (U^*,θ_∞) = c_β( [θ_∞^β] )^1/1-α.

We turn our attention to (λ^s,θ_∞).
We note that, conditionally to θ_∞ = θ, it holds

    lim_T ↑∞1/T[ ∫_0^T (X^λ^s_t)^αθ^β dt - q λ^s_T |θ_∞ = θ]  = C(a(θ),θ^β),   ρ-a.e. θ∈_+,

where C(a,p) is given by (<ref>).
To see this, it is enough to recall that, by Proposition <ref>, for ρ-a.e. θ in _+, we have p^s_t(dx,θ) → p^s_∞(dx,θ) weakly as t →∞, with p^s_∞(dx,θ) given by (<ref>).
Since, conditionally to θ_∞ = θ, the control λ^s is a reflection at the barrier a(θ), we apply <cit.> to get the equality above.
By applying Lemma <ref> with a = a(θ_∞), and exploiting square-integrability of θ_∞, at any time T > 0 we can bound the left hand-side with C(1 + θ_∞^2), for some positive constant C independent of θ_∞.
Therefore, by dominated convergence theorem, we can exchange limit and expectation, to get

    (λ^s,θ_∞) = lim_T ↑∞1/T[ [ ∫_0^T (X^λ^s_t)^αθ_∞^β dt - q λ^s_T |θ_∞]  ]  = [C(a(θ_∞),θ_∞^β)] 
        = 2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ + σ^2)^α[θ_∞^α+β] - qδ[θ_∞].

By rearranging the terms, we have that (U^*,θ_∞) ≤(λ,θ_∞) if and only if equation (<ref>) is satisfied.



Finally, consider a correlated stationary strategy (Z,θ_∞,λ^s) in the family 𝒢^s and define a correlated strategy profile for the N-player game starting from it: we take Z = θ_∞ as correlation device, and, for any i ≥ 1, we consider the policy λ^i,s = (λ^i,s_t)_t ≥ 0^- according to which the state X^i,s is reflected upward at the random barrier a(θ_∞) given by (<ref>).
As in the case of a regular recommendation, for each i ≥ 1, the triple (X^i,s,λ^i,s,θ_∞) has the same law as (X,λ^s,θ_∞), and the processes (X^i,s)_i ≥ 1 are conditionally independent given θ_∞.


Let (Z,θ_∞,λ^r) be a CCE in the class 𝒢^s.
Let λ^N = (λ^i,s)_i=1^N, with λ^i,s the policy according to which the state is reflected upward at the random barrier a(θ_∞) given by (<ref>).
Then, for any c > 0, the correlated strategy profile (θ_∞, λ^N) defines an ε_N-CCE for the N-player game within the set of strategies _N,c, with ε_N → 0 as N→∞.

We omit the proof since it is completely analogous to the proof of Theorem <ref>: it is enough to repeat the proof of Theorem <ref>, invoking Lemma <ref> instead of Lemma <ref>.




 §.§ Nash Equilibrium for the Ergodic Mean-field Game


Since the MFG considered here does not satisfy the assumptions of <cit.>, we cannot directly deduce existence and uniqueness of NE for the MFG can not be applied.
Nevertheless, we have the following result:

If α + β≠ 1, there exists a unique Nash equilibrium (ν^*, θ^*) of the ergodic MFG.
Moreover, the process ν^* reflects the state process at a barrier a^*, and the pair (a^*, θ^*) is given by

    a^* = ( 2δ + σ^2/2δ)^β/1-α - β2α/q(2δ + σ^2(1 - α))^1/1-α-β, 
       θ^* = ( 2δ + σ^2/2δ)^1-α/1-α - β2α/q(2δ + σ^2(1 - α))^1/1-α-β.

If α + β = 1 and the relation

    1 + σ^2/2δ = ( qδ/α)^1/1-α( 1 +(1-α) σ^2/2δ)^1/1-α

holds, then there exist infinitely many mean-field Nash equilibria given by the pair (ν^a(θ),θ), with ν a reflection at the level a(θ) = Kθ^β and θ>0; otherwise, it does not exist any Nash equilibrium for the MFG.


Fix θ > 0.
By applying Lemma <ref> with p=θ^β and λ = 0, the payoff functional (ν,θ) is maximized by the strategy ν^θ which reflect the process X^ν^θ upwards à la Skorohod at the point a(θ) given by 

    â(θ) = 2α/q(2δ + σ^2(1 - α))^1/1-αθ^β/1-α.

In view of (<ref>), in order to get the consistency condition (<ref>) satisfied, we impose

    θ^* =  2δ/2δ + σ^2 K (θ^*)^β/1-α.

If α + β≠ 1, this is equivalent to

    (θ^*)^1 -α -β/1-α = 2δ + σ^2/2δK.

If α + β≠ 1, the function g(θ):=θ^1-α -β/1-α is always non negative, strictly monotone and with image equal to _+, which implies that there exists a unique θ^* so that the above equality is verified; by direct computation, it can be verified that θ^* is given by (<ref>).
Finally, if α + β = 1, condition (<ref>) becomes

    θ^* = 2δ + σ^2/2δ K θ^*.

Thus, the MFG admits infinitely many Nash equilibria if 2δ + σ^2/2δ K = 1, and none otherwise.
By explicit calculations, this is equivalent to 

    2δ + σ^2/2δ = ( qδ/α)^1/1-α( 2δ+σ^2/2δ - ασ^2/2δ)^1/1-α.

By rearranging the terms, we get to (<ref>).




Analogous considerations as in Remark <ref> hold for the MFC problem as well.
On top of those considerations, we note that, when α + β = 1 and condition (<ref>) is not satisfied, we do not have existence of a Nash equilibrium for the ergodic MFG, while the optimality conditions for both classes 𝒢^r and 𝒢^s are still valid.
The ultimate reason is that the procedure outlined in Section <ref> does not involve the usual two steps scheme used to compute mean-field NEs: first, optimize with a fixed flow of moments and, second, perform a fixed point argument to determine the flow. Actually, we first impose the consistency condition and then we restate the optimality condition.


We notice that the pair (ν^*,θ^*) is a correlated stationary strategy in 𝒢^s with deterministic correlation device.
In particular, it satisfies the optimality condition (<ref>).
As a consequence of Theorem <ref>, we also deduce that every NE for the ergodic MFG induces a sequence of approximate Nash equilibria with vanishing error in the N-player game:

Let (ν^*,θ^*) be a Nash equilibrium for the MFG, as given by Proposition <ref>.
For any i ≥ 1, let ν^i,* be the policy according to which the state is reflected upward at the random barrier a^* given by (<ref>).
Then, for any c > 0, the open-loop strategy profile ν^*,N = (ν^i,*)_i = 1^N defines an ε_N-NE for the N-player game within the set of strategies _N,c, with ε_N → 0 as N→∞.


It is enough to notice that, when starting from the NE (ν^*,θ^*) the recommendation λ^N defined in Theorem <ref> is actually an open-loop strategy profile for the N-player game.





§ NUMERICAL ILLUSTRATIONS

In this Section, we numerically illustrate our previous findings.
In particular, we exhibit possible choices of distribution of θ_∞ so that the correlated stationary strategies (Z,θ_∞,λ^r) in 𝒢^r and (Z,θ_∞,λ^s) in 𝒢^s are mean-field CCEs, i.e., according to Propositions <ref> and <ref>, the inequalities (<ref>) and (<ref>) respectively are verified.

We suppose that θ_∞ is distributed as a Gamma with u>0 and scale parameter v > 0.
Then, for any k ≥ 0 the k-th moment of θ_∞∼Γ(u,v) is given by

    [θ_∞^k] = 1/Γ(u) v^u∫_0^∞ x^kx^u-1e^-x/vdx = Γ(u + k)/Γ(u) v^k.

By assuming Gamma distribution on θ_∞, the optimality conditions (<ref>) and (<ref>) for regular and singular recommendation become, respectively,

    c_β( Γ(β+u)/Γ(u))^1/1-α v^β/1-α≤ c_α+βΓ(α + β+u)/Γ(u) v^α + β - c_1 u v,  
        c_β( Γ(β+u)/Γ(u))^1/1-α v^β/1-α≤c_α+βΓ(α + β+u)/Γ(u) v^α + β - c_1 u v,

Given the non-linear structure of the optimality inequalities and the intricate dependence on the parameters in the constants c_1, c_β, c_α + β and c_α + β, we limit ourselves to a specific choice of the parameters.
For the sake of illustrations, we fix δ = 0.1 and σ = 0.2.
Notice that this choice satisfies Assumption <ref>.
We also set q = 2.




 §.§ The case α + β < 1

In this case, there exist both a unique mean-field NE and a unique optimal control for the MFC problem.
For the sake of comparison with the payoffs of the NE and the MFC solution, we are also interested in finding values of (u,v) ∈_+^2 so that the reward of the associated CCE is higher than the reward of the NE.
Therefore, we pair equations (<ref>) and (<ref>) with, respectively,

    c_α+βΓ(α + β+u)/Γ(u) v^α + β - c_1 u v ≥c_α+β(θ^*)^α + β - c_1 θ^*, 
       c_α+βΓ(α + β+u)/Γ(u) v^α + β - c_1 u v ≥c_α+β(θ^*)^α + β - c_1 θ^*.

Figure <ref> shows that there exist infinitely many mean-field CCEs both in 𝒢^r and 𝒢^s which yield an higher reward than the Nash equilibrium (ν^*,θ^*).
Here, we set α = 0.3 and β = 0.5.





Figure <ref> shows the reward associated to those mean-field CCEs in 𝒢^r and 𝒢^s that yield an higher reward than the Nash equilibrium (ν^*,θ^*).
The improvement on the Nash equilibrium is ≈ 17% of the reward yielded by the mean-field control solution ν̂ in the singular case, and ≈ 12% in the regular case.
We notice that the payoff associated to the Nash equilibrium for the ergodic MFG is strictly less than the reward given by the solution of the MFC problem.
This can be directly deduced from the fact that the the stationary mean θ̂ associated to the MFC solution ν̂ is the unique maximizer of the function f(θ) given by (<ref>), which can be equivalently expressed as f(θ)=c_α+βθ^α + β - c_1θ.
Since by Proposition <ref> the value of the ergodic MFG at the Nash equilibrium (ν^*,θ^*) can be expressed as f(θ^*), we deduce

    (ν^*,θ^*) = f(θ^*) < f(θ̂) =  (ν̂,[X^ν̂_∞]).

The reward of the MFC solution (ν̂,[X^ν̂_∞]) appears to be an unattainable upper bound for the set of mean-field CCEs payoffs.
While we limit to empirically observe this phenomenon, we point out that it is widely expected and that it is coherent with the findings of <cit.> for linear-quadratic mean-field games.









 §.§ The case α + β = 1

In this case, by Theorem <ref>, either there does not exist any mean-field NE or there exist infinitely many, depending on whether the relation (<ref>) is satisfied.
As noticed in Remark <ref>, the optimality inequalities (<ref>) and (<ref>) are still valid.
By imposing β = 1 - α, and noticing that the parameter v is not anymore relevant in the inequalities, equations (<ref>) and (<ref>) can be written in terms of u only:

    c_β( Γ(1- α +u)/Γ(u))^1/1-α≤( c_α+βΓ(u) - c_1) u ,  
        c_β( Γ(1-α+u)/Γ(u))^1/1-α≤( c_α+βΓ(u) - c_1) u.

We observe that there exist  maximal values u^*_r and u^*_s, depending on α, so that the inequalities (<ref>) and (<ref>) are verified by any 0 < u ≤ u^*_r and 0 < u ≤ u^*_s, respectively.
Figure <ref> plots such maximal values u^*_r and u^*_s as functions of α∈ (0,1).
Therefore, we observe existence of mean-field CCEs even in the case in which there does not exist any mean-field NE.
We notice that, when considering correlated stationary strategies (Z,λ^s,θ_∞), there exist a value α̅ so that that the inequality (<ref>) is verified for any u > 0, i.e. u^* = ∞.
It can be numerically shown that such value α̅ is the unique solution of (<ref>), for fixed δ, σ and q.
We can explain this phenomenon as follows: for α = α̅, by Theorem <ref>, for any θ>0 the pair (ν^a(θ),θ) is a mean-field NE, where ν^a(θ) is the policy that reflects the process X^ν^a(θ) upwards at the level a(θ) = 2δ/2δ + σ^2θ.
Therefore, any correlated stationary strategy (Z,λ^s,θ_∞) ∈𝒢^s is just a randomization, or a mixture, of mean-field NE, since λ^s reflects the process X^λ^s at the same barrier a(θ_∞) = 2δ/2δ + σ^2θ_∞.
To put in other terms, the pair (a(θ_∞),θ_∞) is supported on the set of mean-field NEs.
This implies that the optimality condition is satisfied by any θ_∞ so that the optimality inequality (<ref>) holds true, and so by any (Z,λ^s,θ_∞) ∈𝒢^s.








 §.§ The case α + β > 1
 The case α + β > 1 is completely analogous to the case α + β < 1. We just observe that, in this case, the MFC problem is ill-posed, in the sense of Theorem <ref> and therefore a-priori we do not have any upper-bound on the set of mean-field CCEs payoffs.







tocsectionAppendix


§ APPENDIX




 §.§ Control-theoretic Results



Point <ref> follows from direct calculations.
As for point <ref>, the solution of the Skorohod problem follows from standard arguments (see, e.g. <cit.>).
As for ergodicity, note that the derivative of the speed measure of the process X^ν^a is given by m'(x) = 2/σ^2 x^ - 2δ/σ^2_[a,∞)(x), which is integrable over [0,∞) for any a>0.
By <cit.>, the process X^ν^a is ergodic and admits 1/m([a,∞))m'(x) dx = p_a(dx) given by (<ref>) as the unique invariant distribution.

As for <ref>, set L=1+a and observe that, since a < L and the control ν^a never acts when the process lies in the region {x:   x > a}, it holds ( dν^a ) ∩{X^ν^a≥ L } = ∅, -a.s..
The proof of <cit.> implies that

    [| X^ν^a_t | ^2] ≤ 2L^2 + [ξ^2].

By definition of L and Assumption <ref>, the estimate follows.
Finally, By (<ref>), we have, for any T > 0

    ν^a_T = X^ν^a_T - ξ +δ∫_0^T X^ν^a_s ds -∫_0^T σ X^ν^a_s dW_s.

By taking the expectation, applying Jensen inequality and taking advantage of Itô isometry, we find

    1/T^2[|ν^a_T | ] ≤c/T^2( 1 + [| X^ν^a_T |^2] + [ ( ∫_0^T  X^ν^a_s  ds )^2 ] +  [ ( ∫_0^T X^ν^a_s dW_s )^2 ] ) 
       ≤c/T^2( 1 + (T + T^2)sup_T ≥ 0[(X^ν^a_T)^2]  ) ≤ c(1 + a^2),

where last inequality follows from previous estimate.




We sketch the proof, which is essentially similar to the proof of <cit.>.
Recall from <ref> the definition of the function g(x,p,λ).
Let 𝒯 be the set of 𝔽-stopping times.
Consider the auxiliary optimal stopping problem

    u(x,p,λ):= inf_τ∈𝒯_x [ ∫_0^τ e^-δ t  g_x(X_t, p,λ) dt  + q e^-δτ],

where X̂=(X̂_t)_t ≥ 0 is defined by

    dX̂_t=(-δ + σ^2 )X̂_t dt + σX̂_tdW_t,

and _x[·] denotes [·|X_0 = x], x ∈_+.
Let ϕ̂_0 is the non-increasing fundamental solution of

    1/2σ^2 x^2 u_xx(x) + (-δ + σ^2) x u_x(x) -δ u(x) = 0,

and let m̂' be the density of the speed measure m̂ of the process X̂.
By the same reasoning of <cit.>, it can be shown that, if there exists a unique a^*=a^*(p,λ) > 0 solution to

    ∫_a^*^+∞ϕ̂_0(y)( α p y^α - 1 - (qδ - λ) ) m̂'(y) dy = 0,

then the value function u(·,p,λ) is C^1(ℝ_+) with u_xx(·,p,λ) ∈ L^∞_loc(_+), and that the optimal stopping time is given by τ̂(x,p,λ):=inf{t≥ 0   |  X̂_t ≤ a^*(p,λ) }. 

By explicit calculations, we have ϕ̂_0(y) = y^-1 and m̂'(y) = 2/σ^2 y^ - 2δ/σ^2, so that (<ref>) becomes 

    ∫_a^*^+∞( α p y^α - 1 - (qδ - λ) )y^-2δ/σ^2 - 1 dy = 0.

Since, by assumption, qδ - λ > 0, there exists a unique solution a^*(p,λ) given by (<ref>).
The proof can be then completed by the same methods of Step B.1 in <cit.>.




We deal with the case q'<∞.
The case q'=∞ is completely analogous.
We start by proving <ref>.
Let ν̂ be optimal for the control problem with dynamics (<ref>) and payoff functional J(·,p,λ).
Recall the definition of the function g(x,p,λ) in (<ref>).
For any ν∈_mf, ∈ (0,1/2], set ν^ = ν + (1-)ν̂.
Set

    f(,T)     = 1/(  1/T[ ∫_0^T g(X^ν^_t,p,λ) dt - q ν^_T ] -  1/T[ ∫_0^T g(X^ν̂_t,p,λ)dt - q ν̂_T]  ).

For any ∈ (0,1/2] and T > 0, it holds

    f(,T)      = 1/1/T( [ ∫_0^T ( ∫_0^1 g_x(X^ν̂_t + τ (X^ν^_t - X^ν̂_t ),p,λ)dτ) (X^ν^_t - X^ν̂_t ) dt - q (ν^_T - ν̂_T) ] ) 
        = 1/T( [ ∫_0^T ( ∫_0^1 g_x(X^ν̂_t + τ (X^ν^_t - X^ν̂_t ),p,λ)dτ) (X^ν_t - X^ν̂_t) dt - q ( ν_T - ν̂_T) ] ).


We claim

    lim_↓ 0 f(,T) = 1/T[ ∫_0^T g_x(X^ν̂_t,p,λ) (X^ν_t - X^ν̂_t) dt - q (ν_T -ν̂_T ) ]

uniformly in T.
Indeed,

    | f(,T ) - 1/T[ ∫_0^T g_x(X^ν̂_t,p,λ) (X^ν_t - X^ν̂_t) dt - q (ν_T -ν̂_T ) ] |
       ≤|1/T[ ∫_0^T  (X^ν̂_t - X^ν_t ) ∫_0^1 ( g_x(X^ν^_t + τ (X^ν̂_t - X^ν^_t ),p,λ) -  g_x(X^ν̂_t,p,λ)  ) dτ dt  ] |
       ≤1/T[ ∫_0^T  | X^ν̂_t - X^ν_t |∫_0^1 | g_x(X^ν^_t + τ (X^ν̂_t - X^ν^_t ),p,λ) -  g_x(X^ν̂_t,p,λ)  |  dτ dt  ].

By continuity of g_x(x,p,λ) in x, the inner integral converges to 0 as ↓ 0 for any t ≥ 0, -a.s..
By linearity of the dynamics (<ref>), since X^ν_t is positive for any t, it holds

    X^ν^_t + τ (X^ν̂_t - X^ν^_t ) = τ X^ν̂_t + (1-τ)  X^ν^_t ≥1/2X^ν^_t = 1/2( X^ν_t +(1-)X^ν̂_t ≥1/2 X^ν̂_t) ≥1/4X^ν̂_t.

Since | g_xx(y,p,λ) | = α(1-α) p x^α -2≤ c | x|^α-2 for any y ≥ x, g_x(y,p,λ) is Lipschitz on [x,+∞) with Lipschitz constant c | x|^α-2.
Then, it follows

    | f(,T ) - g(T) |≤c/T[ ∫_0^T  | X^ν̂_t - X^ν_t || X^ν̂_t |^α - 2∫_0^1 | X^ν^_t + τ (X^ν̂_t - X^ν^_t ) - X^ν̂_t |  dτ dt  ] 
       ≤ c 1/T[ ∫_0^T | X^ν̂_t - X^ν_t |·| X^ν^_t - X^ν̂_t || X^ν̂_t |^α - 2 dt  ] 
       ≤ c ( sup_T > 01/T[ ∫_0^T | X^ν̂_t |^2q dt  ] + sup_T > 01/T[ ∫_0^T | X^ν_t |^2q dt  ] )sup_T > 0( 1/T[ ∫_0^T | (X^ν̂_t)^α -2|^q' dt  ] )^1/q' ,

where we used Hölder's inequality together with conditions (<ref>) and (<ref>).
On the other hand, by taking the limit with respect to T, it holds

    lim inf_T ↑∞ f(,T) ≤1/(  lim inf_T ↑∞1/T[ ∫_0^T g(X^ν^_t,p,λ) dt - q ν^_T ] .
       . - lim inf_T ↑∞1/T[ ∫_0^T g(X^ν̂_t,p,λ)dt - q ν̂_T]  ) = 1/ ( (ν^) - (ν̂)) ≤ 0,

by using the inequality lim inf_n a_n - lim inf_n b_n ≥lim inf_n (a_n - b_n) and by optimality, for any ∈ (0,1/2].
Lemma <ref> then implies 

    lim inf_T ↑∞1/T[ ∫_0^T g_x(X^ν̂_t,p,λ) (X^ν_t - X^ν̂_t) dt - q (ν_T -ν̂_T ) ] ≤ 0.

This concludes the proof of point <ref>.

As for point <ref>, assume that both (<ref>) and <ref> hold.
Then, by using the inequality lim sup_n z_n -lim inf_n x_n ≥lim sup_n(z_n - y_n) (see <cit.>) and concavity of g jointly in x, it holds

    (ν̂,p,λ) - (ν,p,λ) ≥lim sup_T ↑∞1/T[ ∫_0^T( g(X^ν_t,p,λ) - g(X^ν̂_t,p,λ)) dt - q(ν̂_T - ν_T) ] 
       ≥lim sup_T ↑∞1/T[ ∫_0^T g_x(X^ν_t,p,λ)(X^ν̂_t - X^ν_t) dt - q(ν̂_T - ν_T) ] ≥ 0,

which implies that ν̂ is optimal within _s.
If instead we assume (<ref>), the claim follows from sublinearity of the inferior limit.




We verify the identity by explicit calculations.
First notice that

    λ(θ)     = q _uδ - 2δ+σ^2/2δ^1- α2δα/2δ + σ^2(1-α)θ^α + β - 1 = qδ - αlim inf_T ↑∞1/T[ ∫_0^T (X^ν^a(θ)_t)^αθ^β - 1].

On the other hand, we have

    f'(θ)     = -q δ + (α + β )2δ + σ^2/2δ + σ^2(1-α)( 2δ/2δ+σ^2)^αθ^α+β-1
        = -qδ + (α + β )lim inf_T ↑∞1/T[ ∫_0^T (X^ν^a(θ)_t)^αθ^β - 1].

By summing the two terms, we find

    f'(θ) + λ(θ) = βlim inf_T ↑∞1/T[ ∫_0^T (X^ν^a(θ)_t)^αθ^β - 1] = lim inf_T ↑∞1/T[∫_0^T π_θ(X^ν^a(θ)_t,θ)dt ].







 §.§ Integrability Results



Let θ > 0 and let m'_∞,θ(x) be given by (<ref>).
For any k ≥ 0, the ∫_0^∞ x^k m'_∞,θ(x)dx is finite if and only if 2δ - (k-1)σ^2 > 0.
If so, it holds

    ∫_0^∞ x^k m'_∞,θ(x)dx = θ^k - 2δ/σ^2 -1 2δ/σ^2^k -1 - 2δ/σ^2Γ2δ/σ^2 - k + 1.



Let k ≥ 0 so that 2δ - (k-1)σ^2 > 0.
By setting z=xθ in the integral in (<ref>), we have:

    ∫__+   x^k x^ -2δ/σ^2  - 2exp -2δ/σ^2θ/x dx = θ^k - 2δ/σ^2 -1 ∫_0^∞ z^k - 2δ/σ^2 -2 exp -2δ/σ^21/zdz,

and, by making the change of variables t =2δσ^2·1z, we have

    ∫_0^∞ z^k - 2δ/σ^2 -2 exp -2δ/σ^21/zdz = 2δ/σ^2^k -1 - 2δ/σ^2∫_0^∞ t^2δ/σ^2 -k e^-tdt = 2δ/σ^2^k - 1 - 2δ/σ^2Γ2δ/σ^2 - k + 1,

which yields (<ref>).




For any θ > 0, let X^θ = (X^θ_t)_t ≥ 0 be the solution of 

    dX^θ_t = δ(θ-X^θ_t)dt + σ X^θ_t dW_t,    X^θ_0 = ξ.

There exists a constant C independent of θ so that it holds

    sup_t ≥ 0[(X^θ_t)^2]  ≤ C(1 + θ^2).



By Itô formula, we have

    d(X^θ_t)^2 = [-(2δ - σ^2)(X^θ_t)^2 + 2δθ (X^θ_t) ]dt + 2 σ  (X^θ_t)^2 dW_t,    (X_0^θ)^2= ξ^2,

so that, by taking the expectation, we get

    [(X^θ_t)^2] = e^-(2δ - σ^2) t( [ξ^2] + 2 δθ∫_0^t [X^θ_s] e^-(2δ - σ^2) s ds ).

By (<ref>), we have [X^θ_s] ≤ C(1+θ) for any s ≥ 0 and, by Assumption <ref>, 2δ - σ^2>0.
Thus, it holds 

    [(X^θ_t)^2]    ≤ e^-(2δ - σ^2) t( [ξ^2] + C(1 + θ)  2 δθ∫_0^t e^(2δ - (n-1)σ^2) s ds ) 
       ≤ C(1 + θ^2) + e^-(2δ - σ^2) t( [ξ^2] - C_1 (1 + θ^2) ) ≤ C(1+θ^2).

This completes the proof.





 §.§ Auxiliary Results for the Backward Convergence Problem



Suppose [θ_∞^2]<∞.
Let (X^i)_i= 1^N be i.i.d. as Y conditionally to θ_∞, with either Y=X^λ^r or Y=X^λ^s, and let κ(dx,θ) be equal to p^r_∞(dx,θ) given by (<ref>) or equal to p^s_∞(dx,θ) given by (<ref>).
Let θ^-i,N_t = 1/N-1∑_j ≠ iX^j_t, for t ≥ 0, 1 ≤ i ≤ N.
Then, it holds

    lim_ T ↑∞   1/T∫_0^T [ |( θ^-i,N_t )^β - θ_∞^β|^2 |θ_∞] dt  = ∫__+^N-1|( 1/N-1∑_j ≠ ix_j )^β - θ_∞^β|^2 ⊗_j ≠ iκ(dx_j,θ_∞),  -a.s.



Suppose without loss of generality that (Ω,,) is a Polish space (if it is not, we work on the canonical space).
Consider the regular conditional probability of  given θ_∞. Denote the regular conditional probability of  given θ_∞ = θ by ^θ(·) = (· |θ_∞ = θ), and by ^θ[·] the expectation with respect to the probability measure ^θ.
Conditionally to θ_∞ = θ, we have that (X^j)_j ≠ i are i.i.d. as Y; thus, in particular, the process θ^-i,N is a positively recurrent regular diffusion with ergodic measure ⊗_j ≠ iκ(dx_j,θ) (see, e.g. <cit.>).
By the ergodic ratio theorem, it holds 

    lim_ T ↑∞1/T∫_0^T |( θ^-i,N_t )^β - θ^β|^2 dt = ∫__+^N-1|( 1/N-1∑_j ≠ ix_j )^β - θ^β|^2 ⊗_j ≠ iκ(dx_j,θ),   ^θ-a.s..

Therefore, convergence in probability with respect to the probability measure ^θ holds as well.
In order to get convergence in L^1 as well, we show that the family of random variables in the left hand-side of (<ref>) is uniformly integrable. 
By, e.g., <cit.>, this implies that we can take the expectation with respect to ^θ and exchange the limit and expectation, to get

    lim sup_ T ↑∞   1/T∫_0^T ^θ[ |( θ^-i,N_t )^β - θ_∞^β|^2 ] dt = ∫__+^N-1|( 1/N-1∑_j ≠ ix_j )^β - θ^β|^2 ⊗_j ≠ iκ(dx_j,θ),

holds for ρ-a.e. θ > 0, which is equivalent to (<ref>).
To verify uniform integrability, take r = 1β > 1.
By standard estimates, we have

    ^θ   [ |1/T∫_0^T |( θ^-i,N_t )^β - θ^β|^2 dt |^r ] ≤2^2r -1/T∫_0^T ( θ^2 r β + ^θ[ ( θ^-i,N_t )^ 2 r β] ) dt 
       ≤ C (  θ^2 + sup_t ≥ 0^θ[| Y_t |^2] ) ≤ C (1+θ^2),

where last inequality holds true by Lemma <ref> if Y = X^λ^r, and by Lemma <ref> if Y = X^λ^s.
This implies that such family is bounded in L^r-norm, thus, since r > 1, uniformly integrable.





Let κ(dx,θ) be either equal to p^r_∞(dx,θ) given by (<ref>) or equal to p^s_∞(dx,θ) given by (<ref>).
Then, for any θ > 0, it holds

    lim_N →∞∫__+^N-1|( 1/N-1∑_j ≠ ix_j )^β - θ^β|^2⊗_j ≠ iκ (dx_j,θ) = 0.



Let (Y_i)_i ≥ 1 be a sequence of i.i.d random variables with law κ(dx,θ), defined on some probability space (X,𝒳,μ).
Up to reindexing, the integral in (<ref>) can be expressed in terms of the expectation of a function of Y̅^n = 1n∑_i=1^n Y_i.
Since (Y^i)_i ≥ 1 are i.i.d. as κ(dx,θ) and integrable, we have

    (Y̅^n)^β→θ^β  μ-a.s.

by the strong law of large number and continuity of the function x ↦ x^β.
Therefore, convergence in probability holds as well.
To conclude, we show that the sequence ( | (Y̅^n)^β - θ^β|^2 )_n ≥ 1 is uniformly bounded in L^r-norm, for some r>1, which implies that the sequence is uniformly integrable, and thus the convergence in L^2-norm holds by, e.g., <cit.>.
Let r = 1β > 1.
By standard estimates, we have

    [ (| (Y̅^n)^β - θ^β|^2)^r ] ≤ 2^2 r - 1 ( θ^2 + [(Y̅^n)^2] ) ≤ C( θ^2 +  [Y_1^2] ),

where in the last inequality we used the identical distribution of the sequence (Y_i)_i ≥ 1.
The expectation of Y_1^2 is finite by Lemma <ref> if κ = p^r_∞ and by Lemma <ref> if κ = p.
This concludes the proof.





 §.§ A Technical Result on the Exchange of Limits



Let (a_n,m)_n,m ≥ 1 be a real valued sequence. Suppose that the following holds:

    
  *  lim_n →∞ a_n,m = b_m uniformly in m, and
    
  * lim inf_m →∞ a_n,m = c_n for every n ≥ 1.

Then, it holds

    lim inf_m →∞lim_n →∞ a_n,m≤lim_n →∞lim inf_m →∞ a_n,m.



For any n ≥ 1, consider a subsequence (a_n,m_k)_n,k ≥ 1 so that c_n = lim inf_m →∞ a_n.m = lim_k →∞ a_n,m_k.
Since <ref> is satisfied by the subsequence (a_n,m_k)_n,k ≥ 1 as well, Moore-Osgood theorem implies that there exists A ∈ so that

    lim_n →∞lim_k →∞ a_n,m_k = A = lim_k →∞lim_n →∞a_n,m_k.

In particular, (<ref>) implies that (b_m_k)_k ≥ 1 is a convergent subsequence of (b_m)_m ≥ 1.
Therefore, we have

    lim inf_m →∞lim_n →∞ a_n,m = lim inf_m →∞ b_m ≤lim_k →∞b_m_k = lim_k →∞lim_n →∞ a_n,m_k =  lim_n →∞lim_k →∞a_n,m_k = lim_n →∞lim inf_m →∞ a_n,m ,

where last equality holds by definition of (a_n,m_k)_n,m ≥ 1. This concludes the proof.









 §.§ Acknowledgements

The authors acknowledge the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project-ID 317210226 - SFB 1283.

The first author acknowledges financial support under the National Recovery and Resilience Plan (NRRP), Mission 4, Component 2, Investment 1.1, Call for tender No. 1409 published on 14.9.2022 by the Italian Ministry of University and Research (MUR), funded by the European Union - NextGenerationEU– Project Title Probabilistic Methods for Energy Transition - CUP G53D23006840001 - Grant Assignment Decree No. 1379 adopted on 01/09/2023 by the Italian Ministry of Ministry of University and Research (MUR).




 §.§ Disclosure statement
 The authors declare that none of them has conflict of interests to
mention.



abbrv




