Evolutionary dynamics in the public goods games with switching between punishment and exclusion
Linjie Liu Shengxian Wang Xiaojie Chen Matjaz Perc


APS/123-QED








xiaojiechen@uestc.edu.cn
School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 611731, China


matjaz.perc@uni-mb.si
Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, SI-2000 Maribor, Slovenia
Complexity Science Hub, Josefstädterstraße 39, A-1080 Vienna, Austria
School of Electronic and Information Engineering, Beihang University, Beijing 100191, P.R. China


Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors and players adopting the switching strategy in a well-mixed population, and this regardless of whether the pool-based or the peer-based switching strategy is introduced. Moreover, we show that the pure exclusion strategy alone is able to evoke a limit cycle attractor in the evolutionary dynamics, such that cooperation can coexist with other strategies.


Evolutionary dynamics in the public goods games with switching between punishment and exclusion
    Matjaž Perc
    December 30, 2023
===============================================================================================



Large-scale cooperation among unrelated individuals distinguishes humans markedly from other animals, and it is indeed crucial for our evolutionary success. Cooperation is remarkable because it is costly for the individual that cooperates, but it is beneficial for the society as a whole. As such, to cooperate is in contradiction with the Darwinistic principle of maximizing personal fitness, and it is therefore challenged by defection. Individuals are thus torn between what is best for them and what is best for the society – the blueprint of a social dilemma. The theoretical framework for studying this fascinating aspect of our biology is evolutionary game theory, with the most commonly used games being the prisoner's dilemma and the public goods game. The latter two games describe the essence of the problem succinctly for pairwise and group interactions, respectively. The resolution of social dilemmas towards the pro-social outcome has received ample attention in the recent past. In our paper, we extend the scope of the classic public goods game with cooperators and defectors to account for a new third type of strategy, namely the switching strategy that either punishes or excludes defectors depending on their numbers. Our research reveals fascinatingly different evolutionary dynamics, including the stable coexistence of three strategies and limit cycles, which enables cooperators to survive where otherwise they would perish. These results have important implications for the better understanding of cooperation, and we also hope they will be inspiring for economic experiments in the future.



§ INTRODUCTION

The emergence of altruistic behavior among unrelated individuals has been a puzzling phenomenon, since such behavior is usually costly to perform but benefits others
<cit.>.
A number of game theoretic researches over the past decades have provided numerous answers to this question, such as indirect reciprocity, reputation, reward, and punishment <cit.>. Among them, scholars pay more attention to the role of pro-social punishment played in promoting public cooperation <cit.>. Theoretical and experimental studies have indicated
that pro-social punishment can reduce the expected payoff of self-interested individuals since they need to pay an extra fine <cit.>. However, such action is also costly, it results in the second-order free-riders dilemma, in which individuals may prefer to benefit from punishment but do not contribute the related costs
<cit.>.

Social exclusion has recently drawn more attention since it can perform better than costly punishment for maintaining cooperation  <cit.>.
It is thought that social exclusion is still an advantageous strategy even facing with a large number of free-riders,
due to the fact that excluding defectors from sharing benefit can decrease the number of beneficiaries.
Recently, such exclusion strategy has been studied from an evolutionary perspective by Sasaki and Uchida <cit.>. The results show that social exclusion strategy can overcome two difficulties of costly punishment: first, rare punishers can not defeat a large area of free-riders; second, punishers will be eliminated by natural selection in the presence of second-order free-riders. Subsequently, Liu et al. <cit.> studied the competition between pro-social exclusion and punishment in finite populations, and claimed that social exclusion can always do better than punishment.

It is worth pointing out that in most previous studies the evolution of these two incentive strategies has been explored in a manner in which the two strategies work independently <cit.>. But in the realistic world, what is pretty widespread is the combined use of these sanctioning strategies. It is still unclear how such combined strategy can promote cooperation from the theoretical perspective. Considering the different roles of pro-social punishment and exclusion played in raising cooperation, it is interesting to investigate how to jointly use the two strategies for the promotion of cooperation. Notice that a specific example in the realistic world is the problem of environmental pollution control. If the number of enterprises discharging pollutants illegally exceeds a given threshold, the Environmental Protection Agency (EPA) will force these enterprises to suspend operations. Otherwise the EPA will impose fines on them <cit.>.

In this paper, we thereby propose a switching strategy with which individuals can either punish or exclude free-rider in the public goods game (PGG) depending on the number of defectors in the group. Specifically, if the number of defectors in the group is above a certain threshold, individuals who contribute to the monitoring organization will behave as excluders, otherwise they act as punishers. We consider these assumptions and then construct a model based on the one in our previous work <cit.>. Furthermore, in our model we further consider that strategy-switching individuals need to pay a monitoring cost, which is ignored in Ref. <cit.>, but it is more reasonable for real-world systems. This is because that monitoring the whole population and knowing the number of free-riders in the population require a certain monitoring cost. We find that a middle threshold can make the system converge to a stable coexistence state of cooperators, defectors, and strategy switching players no matter whether pool-based switching strategy or peer-based switching strategy is used. In addition, we prove that when pure exclusion strategy is considered into public goods games, the population system can exhibit a limit cycle where cooperative strategy can coexist with other types of strategies.



§ MODEL AND METHOD




 §.§ Public goods game


We consider an infinite well-mixed of individuals who play the public
goods game. In a group of N individuals, each player has the
opportunity to cooperate by contributing to the common pool with a cost c to itself, or act as defectors by contributing nothing.
Then the sum of all contributions in each group is
multiplied by an enhancement factor r (1<r<N) and equally
distributed among all the N individuals. Thus if all individuals choose to cooperate, the group can yield the maximum benefit rc-c for each player.
If all choose to defect, the group can get nothing.



 §.§ Switching strategy of pro-social punishment and exclusion


We introduce a switching strategy to depict the combined use of pro-social punishment and exclusion.
Different forms of punishment strategies can be chosen based on whether the number of defectors in the group exceeds the tolerance threshold.
Concretely, if the number of free-riders in the group exceeds a given threshold T, those players with the switching strategy will exclude free-riders, otherwise punish them.
Thus, the levels of tolerance threshold are determined by the number of defectors in the group, namely, T=0, ⋯, N.
In particular, T=0 means that all free-riders will be excluded from sharing public goods once there exist strategy-switching players,
while T=N means that all defectors will be punished.
Here, we introduce two switching strategy, namely, peer-based switching and pool-based switching. It is necessary to point out that regardless of peer-based switching strategy or pool-based switching strategy, strategy switching individuals need to bear a monitoring cost τ for detecting the number of defectors in the group before they exclude or punish free-riders.




  §.§.§ Peer-based switching strategy


Here we consider peer-based switching strategists who not only contribute to the common pool but also monitor the number of free-riders in the group.
If the level of defection exceeds a certain level in the group, they become excluders who prevent defectors collecting benefit from the
public goods sharing at a cost c_E on every defector in the group, otherwise they change to be punishers who impose a fine β on each free-rider at a cost γ for themselves <cit.>.
Then the payoffs of pure cooperators (C), pure defectors (D), and peer-based switching strategists (I_E) obtained from the group can be respectively given by


    π_C   =   {   rc-c,    if N_I_E≠0 and N_D≥ T;
       rc(N_C+N_I_E+1)/N-c,       otherwise.
    π_D   =   {   0,    if N_I_E≠0 and N_D≥ T;
       rc(N_C+N_I_E)/N-N_I_Eβ ,      otherwise.
    π_I_E   =   {   rc-c-N_Dc_E-τ,    if N_D≥ T;
       rc(N_C+N_I_E+1)/N-c-N_Dγ-τ ,     otherwise.


where N_C, N_D, and N_I_E denote the numbers of cooperators, defectors, and peer-based switching strategists among the other N-1 players, respectively.



  §.§.§ Pool-based switching strategy


Different from peer-based switching strategy, pool-based switching strategists resort to the institution
of monitoring which can choose to exclude defectors or punish them by giving a corresponding fine B. The costs of exclusion and punishment are δ and G, respectively. Accordingly, the payoffs of cooperators, defectors, and pool-based switching strategists (I_F) obtained from the group can be respectively written as follows.


    π_C   =   {   rc-c,    if N_I_F≠0 and N_D≥ T;
       rc(N_C+N_I_F+1)/N-c,       otherwise.
    π_D   =   {   0,    if N_I_F≠0 and N_D≥ T;
       rc(N_C+N_I_F)/N-N_I_FB ,      otherwise.
    π_I_F   =   {   rc-c-δ-τ,    if N_D≥ T;
       rc(N_C+N_I_F+1)/N-c-G-τ ,       otherwise.


where N_C, N_D, and N_I_F denote the numbers of cooperators, defectors, and pool-based switching strategists among the N-1 players, respectively.




 §.§ Replicator dynamics


In a well-mixed population the fraction of C, D, and I (I_E or I_F) players can be denoted by x, y, and z, respectively.
Thus, x, y, z≥0 and x+y+z=1. Consequently,
the strategy evolution can be studied by using replicator
equations <cit.>

    {ẋ   =x(P_C-P̅),  
    ẏ   =y(P_D-P̅),  
    ż   =z(P_I-P̅),
    .

where P_C, P_D, and P_I represent the expected payoffs of C, D, and I, respectively, and P̅=xP_C+yP_D+zP_I represents the average payoff of the whole population. And we have

    P_i   =   ∑_N_C=0^N-1∑_N_D=0^N-N_C-1N-1N_CN-N_C-1N_D
          x^N_Cy^N_Dz^N-N_C-N_D-1π_i,

where i=C, D, or I.

To better characterize the evolutionary dynamics of the population for different switching threshold, we present our results regarding the switching threshold in the form of bifurcation diagrams in the following section. In addition, we provide detailed theoretical analysis when the switching threshold is set to N or 0. Unless otherwise specified, theoretical analyses in special conditions are presented in Appendix A, B, C, and D, respectively.



§ RESULTS



 §.§ Evolutionary dynamics in the population with peer-based switching strategy


We first present the results of evolutionary dynamics in the population with peer-based switching strategy for
different values of T, as shown in Fig.<ref>. Clearly, if level of tolerance is strong enough (T=N), players will opt for defection
(more detailed theoretical analysis is shown in the Appendix A.1).
Besides, an unstable equilibrium point can appear on edge I_ED (see Fig.<ref>(a) and (b)). It needs to stress that a specific stable point appears on the edge I_ED when we decrease threshold value slightly (see Fig.<ref>(b)).
For an intermediate threshold (T=3), a new dynamical characteristic appears, that is, an interior stable equilibrium point displays the simplex S_3.
With decreasing T, the interior stable point moves along a straight line approximately parallel to the edge CD (see Fig.<ref> (c), (d), and (e)).
If T decreases still further to T=0, peer-based switching strategist will always use the exclusion action,
Interestingly, defectors will be excluded by peer excluders, defectors dominate cooperators, and cooperators invade peer excluders, forming a heteroclinic cycle on the boundary of simplex S_3. In order to judge the stability of this heteroclinic cycle, we calculate the eigenvalues of the Jacobian matrix of the three vertex equilibrium points as follows,

    {λ_I_E^-   =-(rc-c-τ),  λ_I_E^+=τ, 
    λ_C^-   =-τ,  λ_C^+=c-rc/N, 
    λ_D^-   =rc/N-c,  λ_D^+=rc-c-τ-(N-1)c_E.
    .

Then we define that λ_I_E=-λ_I_E^-/λ_I_E^+, λ_C=-λ_C^-/λ_C^+, and λ_D=-λ_D^-/λ_D^+, and we have λ=λ_I_Eλ_Cλ_D=rc-c-τ/rc-c-τ-(N-1)c_E>1. Thus the heteroclinic cycle is asymptotically stable <cit.> (see Appendix B.3 for theoretical analysis).
Besides, a stable limit cycle exists in the interior of the simplex S_3 (see Appendix B.4 for theoretical analysis). Frequencies of three strategies oscillate, but the interior equilibrium point is unstable (see Fig.<ref>(f), and more detailed theoretical analysis is presented in the Appendix B.2).


In order to shed light on the details behind the results presented in Fig.<ref>,
we depict the frequency of the mentioned three strategies as a function of time for different switching thresholds in Fig.<ref>. It can be observed that although pure defectors can always occupy the highest proportion of the population for T≥ 2, the evolutionary advantage of defectors is weakened gradually with decreasing T (see Fig.<ref>(a-d)). However, it is worth noting that when T=1, pure cooperators have higher fitness than defectors (see Fig.<ref>(e)). In particular, a periodic oscillation occurs when peer exclusion is performed (see Fig.<ref>(f)), which is corresponding to the stable limit cycle shown in Fig.1(f).



 §.§ Evolutionary dynamics in the population with pool-based switching strategy



We next illustrate how the introduction of pool-based switching strategy influences the cooperation level for different T, as shown in Fig.<ref>.
When T=N, pool-based switching strategists will always act as pool punishers. In this case, no interior equilibrium point appears in the simplex S_3 since P_I_F<P_C, while an unstable equilibrium point exists on edge I_FD with z=N(c+δ+τ)-rc/N(N-1)B (for further details, see Appendix C.1). The system only has one stable point D, which means that defectors will dominate the whole population.
However, this phase portraits will be changed when we reduce the tolerance threshold T. When T=4, an interior stable equilibrium point is present in the simplex S_3, thus cooperators, defectors, and pool-based switching strategists can coexist steadily in the population (see Fig.<ref>(b)). Furthermore, with decreasing T the interior stable equilibrium point moves towards full cooperation state (see Fig.<ref>(b), (c), (d), and (e)). As a special case of pool-based switching strategy, T=0 means that pool-based switching strategists will always act as pool excluders. The unique stable coexistent equilibrium point changes to a center surrounding by periodic closed orbits. The reason is that pool excluders invade defectors, cooperators invade pool excluders, and defectors invade cooperators (see Fig.<ref>(f)). To analyze the dynamics in the interior of S_3, we introduce a new variable ε=x/x+y, which represents the fraction of
cooperators among members who do not contribute to the exclusion pool. This yields

    ε̇   =   -ε(1-ε)(P_D-P_C).

By substituting x=ε(1-z) and P̅=x(P_C-P_D)+(1-z)(P_D-P_F)+P_F into ż=z(P_F-P̅), we have ż=z[x(P_D-P_C)-(1-z)(P_D-P_F)].
Thus we have

    {ε̇   =-ε(1-ε)[(1-z)^N-1rc(N-1)/N-rc+c],
    ż   =z(1-z)[rc-c-δ-τ-ε(rc-c)].
    .

Through detailed theoretical analysis, we can prove that the system is a conservative Hamiltonian system (see Appendix D.3 for details).




To further explain the results presented in Fig.<ref>, we continue by showing the time evolution of the frequencies of these three strategies for different values of T in Fig.<ref>. When T=N, defection strategy can be dominant, which is irrelevant to the initial state (see Fig.<ref>(a)). Furthermore, by decreasing the tolerance level, however, we can observe the fraction of cooperators increases (see Fig.<ref>(b), (c), (d), and (e)). Of particular note is that pure cooperation strategy can become the most advantageous strategy when T is set to an intermediate value, e.g. T=2. As T is decreased further, the advantage of cooperators is further enhanced (see Fig.<ref>(d) and (e)).
Particularly, for T=0 the frequencies of C, D, and I_F display periodic oscillations, which correspond to the limit cycle in Fig.<ref>(f).



§ DISCUSSION


Thus far, many previous theoretical works have revealed that pro-social punishment and exclusion strategies can both maintain sufficiently high levels of public cooperation no matter whether these two strategies are implemented separately or jointly <cit.>. However, few studies have thus far considered the combined use of these two strategies despite it is particularly common in our real society. In this paper, we have introduced the switching strategy with which individuals can either punish or exclude free-riders in the public goods game (PGG), depending on the number of defectors in the group. Based on the evolutionary game theoretical models we have studied the evolutionary dynamics in the well-mixed population with two different switching forms, by focusing particularly on the role of switching threshold played in the evolutionary dynamics of cooperation.


We have shown that a stable coexistence state among cooperators, defectors, and peer-based switching strategists can appear when an intermediate switching threshold is used. In addition, the reduction of switching threshold can enhance the level of public cooperation.
It is necessary to point out that the special cases in our model (T=N and T=0 correspond to peer exclusion and peer punishment, respectively) have been investigated in a recent work <cit.>, which demonstrated that social exclusion strategy can not only avert free-riders, but also prevent second-order free-riders from invading, which solves the two substantial difficulties of peer punishment. Interestingly, we find that the introduction of an observation cost will completely change the evolutionary results of these two strategies in our model. Concretely, the coexistence of cooperators and peer punishers will disappear, and defection becomes a global stability strategy (see Fig.<ref>(a)). Or, the periodic oscillations among the three strategies replace the coexistence of cooperators and peer excluders (see Fig.<ref>(f)). Although similar shapes of oscillations have been presented in previous works <cit.> where oscillations are caused by the feedback between players' payoffs and their local interaction topology or the cyclic dominance of the three species, the mechanisms for the oscillations between our work and the two studies mentioned above are different. In our work, we investigate the evolutionary dynamics among cooperators, defectors, and switching strategists in an infinite population. The oscillation is caused by the mutual restriction among these three strategists, that is, defectors will be excluded by peer excluders, defectors defeat cooperators, and cooperators invade peer excluders.

We have also investigated the evolutionary dynamics of pool-based switching strategy and revealed that in addition to the two special cases (T=0 and T=N correspond to the case of pool punishment and pool exclusion, respectively) mentioned above, the intermediate switching threshold values can guarantee the stable coexistence state of cooperators, defectors, and pool-based switching strategists. It needs to point that the result is stile valid even the cost of exclusion exceeds four times than we given in Fig.<ref>. Furthermore, the decrease of threshold value will weaken the advantage of defectors in evolution. Particularly, if switching threshold is zero, the result shows that there can be isolated periodic orbits and hence cooperators, defectors, and pool excluders can coexist by forming limit cycles (see Fig.<ref>(f)).

Finally, we have to note that the switching strategy proposed in this work can better induce the stable coexistence state for cooperation compared with pure punishment or pure exclusion strategies. Besides, such switching strategy can flexibly manage public goods in different environments, such as punishing exiguous free-riders in favorable surroundings or excluding massive free-riders in extremely unfavorable environments. Thus, the presently discussed strategy offers a simple, but still effective, way on how we can better promote the stable coexistence of different strategies including cooperation.















§ APPENDIX A





 §.§ A.1 Peer punishment


We first study the replicator dynamics for defectors (D), cooperators (C), and peer punishers (W). This corresponds to the special case for peer-based switching strategy with T=N. We denote by x, y, and z the frequencies of C, D, and W, respectively. Thus x, y, z≥0 and x+y+z=1. The evolutionary fate of the population can be modeled by the replicator equations, given as

    {ẋ   =x(P_C-P̅),
    ẏ   =y(P_D-P̅),
    ż   =z(P_W-P̅),
    .

where P_C, P_D, and P_W denote the expected payoffs of these three strategies and P̅=xP_C+yP_D+zP_W describes the average payoff of the entire population.

Accordingly, the expected payoffs of these three strategies can be respectively given by

    P_C   =   rc/N(N-1)(x+z)+rc/N-c,  
    
    P_D   =   rc/N(N-1)(x+z)-(N-1)zβ,  
    
    P_W   =   rc/N(N-1)(x+z)+rc/N-c-(N-1)yγ-τ.


Since P_W<P_C, there is no interior fixed point.
Then we investigate the dynamics on each edge of the simplex
S_3. On the edge C-D we have z=0, resulting in
ẏ=y(1-y)(P_D-P_C)=y(1-y)(c-rc/N)>0. Thus the
direction of the dynamics goes from C to D.
On the edge D-W, since x=0 and y+z=1, we have
ż=z(1-z)(P_W-P_D). Here we assume that 0<c+τ-rc/N+(N-1)γ<(N-1)(γ+β), thus solving P_W=P_D results in z=c+(N-1)γ+τ-rc/N/(N-1)(β+γ), which means that there exists a boundary fixed point on the edge D-W.
On the edge C-W, since y=0 and x+z=1, we have
ẋ=x(1-x)(P_C-P_W)=x(1-x)τ>0, thus the
direction of the dynamics goes from W to C.

Therefore there are four equilibria, namely, three vertex fixed points ((x, y, z)=(0, 0, 1), (1, 0, 0), and (0, 1, 0)) and the boundary fixed point ((x, y, z)=(0, 1-c+(N-1)γ+τ-rc/N/(N-1)(β+γ),c+(N-1)γ+τ-rc/N/(N-1)(β+γ))) in the simplex S_3.



  §.§.§ A.2 The stabilities of equilibria


Here we define

    f(x,y)   =x[(1-x)(P_C-P_W)-y(P_D-P_W)],
    
    g(x,y)   =y[(1-y)(P_D-P_W)-x(P_C-P_W)].

Then the Jacobian matrix of the equation system is

    J=[ ∂f(x,y)/∂x ∂f(x,y)/∂y; ∂g(x,y)/∂x ∂g(x,y)/∂y ],

where

    {∂f(x,y)/∂x   =[(1-x)(P_C-P_W)-y(P_D-P_W)]
       +x[-(P_C-P_W)+(1-x)∂/∂x(P_C-P_W)
       -y∂/∂x(P_D-P_W)],
    ∂f(x,y)/∂y   =x[(1-x)∂/∂y(P_C-P_W)-(P_D-P_W)
       -y∂/∂y(P_D-P_W)],
    ∂g(x,y)/∂x   =y[(1-y)∂/∂x(P_D-P_W)-(P_C-P_W)
       -x∂/∂x(P_C-P_W)],
    ∂g(x,y)/∂y   =[(1-y)(P_D-P_W)-x(P_C-P_W)]
       +y[-(P_D-P_W)+(1-y)∂/∂y(P_D-P_W)
       -x∂/∂y(P_C-P_W)].
    .


Then we have the following conclusion.

Theorem 1.
For 1<r<N, only the fixed point (0, 1, 0) is stable, and the other equilibria (0, 0, 1), (1, 0, 0), and (0, 1-c+(N-1)γ+τ-rc/N/(N-1)(β+γ),c+(N-1)γ+τ-rc/N/(N-1)(β+γ)) are unstable.


Proof.
(1) For (x,y,z)=(0,0,1), the Jacobian is

    J=[               τ               0;               0 c+τ-rc/N-(N-1)β ],

thus the fixed point is unstable since τ>0.

(2) For (x,y,z)=(1,0,0), the Jacobian is

    J=[          -τ -(c+τ-rc/N);           0      c-rc/N ],

thus the fixed point is unstable since 1-r/N>0.

(3) For (x,y,z)=(0,1,0), the Jacobian is

    J=[          rc/N-c               0;       -τ-(N-1)γ rc/N-c-τ-(N-1)γ ],

thus the fixed point is stable since rc/N-c<0.

(4) For (x,y,z)=(0,1-c+(N-1)γ+τ-rc/N/(N-1)(β+γ),c+(N-1)γ+τ-rc/N/(N-1)(β+γ)), the Jacobian is

    J=[ a_11    0; a_21 a_22 ],

where a_11=(N-1)yγ+τ, a_21=y(1-y)β(N-1)-(N-1)y^2γ-τ y, and a_22=y(1-y)(N-1)(β+γ), thus the fixed point is unstable since y(1-y)(N-1)(β+γ)>0 and (N-1)yγ+τ>0.





§ APPENDIX B




 §.§ B.1 Peer exclusion


We now consider another special case for peer-based switching strategy with T=0. Thus the replicator
equations are written as

    {ẋ   =x(P_C-P̅),
    ẏ   =y(P_D-P̅),
    ż   =z(P_E-P̅),
    .

where P̅=xP_C+yP_D+zP_E represents the average payoff of the entire population.


We assume that exclusion never fails. In this condition, we can formalize the expected payoffs as follows

    P_C   =   rc-c-(1-z)^N-1rc(N-1)y/N(1-z),
    
    P_D   =   (1-z)^N-1rc(N-1)x/N(1-z),
    
    P_E   =   rc-c-(N-1)yc_E-τ.


Remark 1: Because of z=1-x-y, the system (<ref>) becomes

    {ẋ   =x[(1-x)(P_C-P_E)-y(P_D-P_E)],
    ẏ   =y[(1-y)(P_D-P_E)-x(P_C-P_E)],
    .

where

    P_C-P_E   =   (N-1)yc_E+τ-(1-z)^N-1rc(N-1)y/N(1-z),
    
    P_D-P_E   =   (1-z)^N-1rc(N-1)x/N(1-z)
       -   rc+c+(N-1)yc_E+τ.



Theorem 2. For [N(r-1)/r(N-1)]^1/N-1(N-1)c_E<rc-c-τ<min{(N-1)c_E, [N(r-1)/r(N-1)]^1/N-1(N-1)c_E+rc-c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E}, the system (<ref>) has five fixed points, namely, (x, y, z)=(0, 0, 1), (1, 0, 0), (0, 1, 0), (0, rc-c-τ/(N-1)c_E, 1-rc-c-τ/(N-1)c_E), and ([N(r-1)/r(N-1)]^1/N-1-τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, 1-[N(r-1)/r(N-1)]^1/N-1).

Proof.
By solving system equations (<ref>), we can easily know that there are three vertex fixed points, namely, (0, 0, 1), (0, 1, 0), and (1, 0, 0). Then solving P_C=P_D results in z=1-[N(r-1)/r(N-1)]^1/N-1. When (r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E>τ, solving P_C=P_E leads to y=τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E. Thus there exists an interior fixed point
([N(r-1)/r(N-1)]^1/N-1-τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, 1-[N(r-1)/r(N-1)]^1/N-1) in the simplex S_3.

Then, we study the dynamics on each edge of simplex S_3.
On the edge E-D, y+z=1 results in ż=z(1-z)(P_E-P_D)=z(1-z)[rc-c-τ-(N-1)yc_E], thus there is an equilibrium y=rc-c-τ/(N-1)c_E for 0<rc-c-τ<(N-1)c_E, otherwise E can perform better than D. On the edge C-E, we have x+z=1 and ẋ=x(1-x)(P_C-P_E)=x(1-x)τ>0, thus the
direction of the dynamics goes from E to C. On the edge C-D, D can defeat C, as presented in Appendix A.1.





 §.§ B.2 The stabilities of equilibria


Theorem 3. In the conditions of Theorem 2, only the fixed point (0, 1, 0) is stable, and the other equilibria (0, 0, 1), (1, 0, 0), (0, rc-c-τ/(N-1)c_E, 1-rc-c-τ/(N-1)c_E), and ([N(r-1)/r(N-1)]^1/N-1-τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, 1-[N(r-1)/r(N-1)]^1/N-1) are unstable.

Proof.
We also use the Jacobian matrix of the system to study the stability of equilibria.

(1) For (x,y,z)=(0,0,1), the Jacobian is

    J=[         τ         0;         0 -(rc-c-τ) ],

thus the fixed point is unstable since τ>0.

(2) For (x,y,z)=(1,0,0), the Jacobian is

    J=[          -τ -(c+τ-rc/N);           0      c-rc/N ],

thus the fixed point is a unstable since 1-r/N>0.

(3) For (x,y,z)=(0,1,0), the Jacobian is

    J=[          rc/N-c               0;            a_21 rc-c-τ-(N-1)c_E ],

where a_21=-(rc/N-rc+(N-1)c_E+τ), thus the fixed point is unstable when rc-c-τ-(N-1)c_E>0, while when rc-c-τ-(N-1)c_E<0 it is stable. Particularly, when rc-c-τ-(N-1)c_E=0, we can prove this fixed point is stable by using center manifold theorem (see Theorem. 4 for detail analysis).

(4) When rc-c-τ-(N-1)c_E<0 there is a boundary equilibrium point (x,y,z)=(0, rc-c-τ/(N-1)c_E, 1-rc-c-τ/(N-1)c_E), then the Jacobian is

    J=[ a_11    0; a_21 a_22 ],

where a_11=(N-1)yc_E-y^N-1rc(N-1)/N+τ, a_21=y^N-1rc/N(N-1)-(N-1)y^2c_E-yτ, and a_22=y(1-y)(N-1)c_E, thus the fixed point is unstable since y(1-y)(N-1)c_E>0.

(5) When τ<rc-c-[N(r-1)/r(N-1)]^1/N-1(N-1)c_E<τ+rc-c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, there is an interior equilibrium point (x,y,z)=([N(r-1)/r(N-1)]^1/N-1-τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, 
τ/(r-1)c/[N(r-1)/r(N-1)]^1/N-1-(N-1)c_E, 1-[N(r-1)/r(N-1)]^1/N-1), we define the equilibrium point as (x^*, y^*, z^*) hereafter. And the elements in the
Jacobian matrix for this equilibrium point are written as


    {∂f/∂x(x^*,y^*)   =x^*[(1-x^*)∂/∂x(P_C-P_E)-y^*∂/∂x(P_D-P_E)],
    ∂f/∂y(x^*,y^*)   =x^*[(1-x^*)∂/∂y(P_C-P_E)-y^*∂/∂y(P_D-P_E)],
    ∂g/∂x(x^*,y^*)   =y^*[(1-y^*)∂/∂x(P_D-P_E)-x^*∂/∂x(P_C-P_E)],
    ∂g/∂y(x^*,y^*)   =y^*[(1-y^*)∂/∂y(P_D-P_E)-x^*∂/∂y(P_C-P_E)],
    .


where


    {∂/∂x(P_C-P_E)   =-(x^*+y^*)^N-3rcy^*(N-1)(N-2)/N,
    ∂/∂y(P_C-P_E)   =(N-1)c_E-(x^*+y^*)^N-3rc(N-1)/N
       [(N-1)y^*+x^*],
    ∂/∂x(P_D-P_E)   =(x^*+y^*)^N-3rc(N-1)/N[(N-1)x^*+y^*],
    ∂/∂y(P_D-P_E)   =(N-1)c_E+(x^*+y^*)^N-3
       rcx^*(N-1)(N-2)/N.
    .


Then we define that p_1=∂f/∂x(x^*,y^*)∂g/∂y(x^*,y^*)-∂f/∂y(x^*,y^*)∂g/∂x(x^*,y^*) and q_1=∂f/∂x(x^*,y^*)+∂g/∂y(x^*,y^*). We know that

    p_1   =   x^*y^*(1-x^*-y^*)[∂/∂x(P_C-P_E)∂/∂y(P_D-P_E)
       -   ∂/∂y(P_C-P_E)∂/∂x(P_D-P_E)]
       =   x^*y^*(1-x^*-y^*)rc(N-1)^3/N(x^*+y^*)^N-2[(x^*
       +   y^*)^N-2rc/N-1]
       <   0,

and

    q_1   =   x^*(1-x^*)∂/∂x(P_C-P_E)+y^*(1-y^*)∂/∂y(P_D-P_E)
       -   x^*y^*[∂/∂x(P_D-P_E)+∂/∂y(P_C-P_E)]
       =   (N-1)c_Ey^*(1-y^*-x^*)
       >   0.


We thus conclude that the Jacobian matrix has a positive eigenvalue. Therefore the interior equilibrium point is unstable.


Theorem 4. For rc-c-τ-(N-1)c_E=0, the equilibrium point (0, 1, 0) is stable.

Proof. Because y = 1 - x - z, the dynamic equations (<ref>) become

    {ẋ   =x[(1-x)(P_C-P_D)-z(P_E-P_D)],
    ż   =z[(1-z)(P_E-P_D)-x(P_C-P_D)],
    .

where

    P_C-P_D   =   rc-c-(1-z)^N-1rc(N-1)/N,
    
    P_E-P_D   =   rc-c-(N-1)yc_E-τ
       -   (1-z)^N-1rc(N-1)x/N(1-z).


We know that (x, z)=(0, 0) is equilibrium point of system (<ref>). Then the Jacobian is

    A=[          rc/N-c               0;               0 rc-c-τ-(N-1)c_E ].

When rc-c-τ-(N-1)c_E=0, we know that the eigenvalues of the Jacobian for the fixed point (x, z)=(0, 0) are 0 and rc/N-c. In this condition, we study the stability of the equilibrium point by further using the center manifold
theorem <cit.>. To do that, we construct a matrix M, whose column elements are the eigenvectors of the matrix A, given as

    M=[ 0 1; 1 0 ].

Let T=M^-1, then we have

    TAT^-1=[      0      0;      0 rc/N-c ].

Using variable substitution, we have

    [ j; i ]=T[ x; z ]=[ 0 1; 1 0 ][ x; z ]=[ z; x ].

Therefore, the system (<ref>) can be rewritten as

    {j̇   =j(1-j)[rc-c-(N-1)(1-j-i)c_E
       -τ-(1-j)^N-1rc(N-1)i/N(1-j)]
       -ji[rc-c-(1-j)^N-1rc(N-1)/N],
    i̇   =i(1-i)[rc-c-(1-j)^N-1rc(N-1)/N]
       -ji[rc-c-(N-1)(1-j-i)c_E-τ
       -(1-j)^N-1rc(N-1)i/N(1-j)].
    .

Using the center manifold theorem, we have that j = h(i) is a center manifold for the above system.
Then the dynamics on the center manifold can be described by

    i̇   =   i(1-i)[rc-c-(1-h(i))^N-1rc(N-1)/N]
       -   h(i)i[rc-c-(N-1)(1-h(i)-i)c_E-τ
       -   (1-h(i))^N-1rc(N-1)i/N(1-h(i))].

We start to try h(i)=O(i^2), thus the system (<ref>) can be expressed as

    i̇=i(1-i)[rc-c-rc(N-1)/N]+O(|i|^3).

By defining m(i)=i(1-i)[rc-c-rc(N-1)/N], we have m^'(i)=(1-2i)[rc-c-rc(N-1)/N]. Since m^'(0)<0,
thus we can judge that i=0 is asymptotically stable. Therefore, we can know the fixed point (i, h(i))=(0, 0) is also stable for the system (<ref>). Accordingly, the fixed point (0, 1, 0) is stable as well <cit.>.



 §.§ B.3 Heteroclinic cycle


In this subsection, we show that there is a stable heteroclinic cycle on the boundary of the simplex CDI_E (see Fig. <ref> (f)). When rc-c-τ-(N-1)c_E>0 and r<N, we know that the three vertex equilibrium points (C, D, and I_E) are all saddle nodes, and the heteroclinic trajectories can display on the three edges (CD, DI_E, and I_EC). All of these guarantee the existence of the heteroclinic cycle on the boundary S_3. In the following, we will prove that the heteroclinic cycle is asymptotically stable.

According to Theorem 3, we can get the eigenvalues of the Jacobian matrix of the three vertex equilibrium points, namely, λ_I_E^-=-(rc-c-τ), λ_I_E^+=τ, λ_C^-=-τ, λ_C^+=c-rc/N, λ_D^-=rc/N-c, and λ_D^+=rc-c-τ-(N-1)c_E. Then we define that λ_I_E=-λ_I_E^-/λ_I_E^+, λ_C=-λ_C^-/λ_C^+, and λ_D=-λ_D^-/λ_D^+, and we have λ=λ_I_Eλ_Cλ_D=rc-c-τ/rc-c-τ-(N-1)c_E>1. Thus the heteroclinic cycle is asymptotically stable <cit.>.



 §.§ B.4 Limit cycle


Next, we prove that a stable limit cycle can exist in the simplex CDI_E (see Fig. <ref> (f)). According to the above theoretical analysis, we know that the interior fixed point of the system is unstable. Now we set a closed domain Γ that contains the interior fixed point on the phase plane of the system (<ref>).
Here we take a straight line l=x+y-b=0, where b is an undetermined constant. In order to determine the direction of the trajectory of the system, we solve the derivative of the equation l=x+y-b. Thus we have

    ∂l/∂t   =   ∂x/∂t+∂y/∂t
       =   x(1-x-y)[(N-1)yc_E+τ-(x+y)^N-1rc(N-1)y/N(x+y)]
       +   y(1-x-y)[(x+y)^N-1rc(N-1)x/N(x+y)-rc+c
       +   (N-1)yc_E+τ]
       =   (1-x-y)[(N-1)yc_E(x+y)+(x+y)τ-(rc-c)y]
       =   (1-b)[(N-1)yc_Eb+bτ-(rc-c)y].


Thus when b is sufficiently large, we know that ∂l/∂t<0. In this case, the straight line l=x+y-b=0, x=0, and y=0 are enclosed in a closed domain Γ that points to the interior of the boundary.

On the straight line x=x^*, we select a point (x^*, y_1), where y_1 is slightly larger than y^*, and the trajectory that passes through (x^*, y_1) surrounds the interior fixed point (x^*, y^*), then intersects with the straight line x=x^* at another point (x^*, y_2) which meets y_2>y^*. Since the interior fixed point (x^*, y^*) is unstable, we have y_2>y_1. The trajectory from (x^*, y_1) to (x^*, y_2) and the line segment y_1y_2 form a closed domain Γ_0 containing the interior fixed point. The trajectories on the boundary of the closed domain Γ_0 are all diverged outward. Thus a ring domain is formed between the boundaries of the closed domains Γ and Γ_0, and the trajectories on the boundary of the inner and outer ring will go to the interior of the domain. Accordingly, we prove that there exist a stable limit cycle in the closed domain Γ <cit.>.



§ APPENDIX C




 §.§ C.1 Pool punishment


Next, we analyze the replicator dynamics for public goods game with pool punishment, which corresponds to the special case for pool-based switching strategy with T=N. Accordingly, there are three strategists, namely, cooperators, defectors, and pool punishers, respectively. Thus the replicator equations can be written as

    {ẋ=x(P_C-P̅),
    ẏ=y(P_D-P̅),
    ż=z(P_V-P̅),
    .

where P̅=xP_C+yP_D+zP_V is the average payoff of the whole population. Then the expected payoffs of these three strategies can be respectively given by

    P_C   =   rc/N(N-1)(x+z)+rc/N-c,  
    
    P_D   =   rc/N(N-1)(x+z)-B(N-1)z,  
    
    P_V   =   rc/N(N-1)(x+z)+rc/N-c-G-τ,

where (N-1)(x+z) denotes the expected number of contributors among the N-1 co-players, and B(N-1)z represents the expected fine on a defector.

Theorem 5. For r<N, the system (<ref>) has four fixed points, namely, (x, y, z)=(0, 0, 1), (1, 0, 0), (0, 1, 0), and (0, 1-N(c+G+τ)-rc/N(N-1)B, N(c+G+τ)-rc/N(N-1)B).

Proof. By solving the system equations (<ref>), we can know that there are three vertex equilibrium points, namely, (x, y, z) = (0, 0, 1), (0, 1, 0), and (1, 0, 0).

There is no interior fixed point since P_V<P_C. Then, we study the dynamics on each edge of simplex S_3.
On the edge V-D, y+z=1 results in ż=z(1-z)(P_V-P_D)=z(1-z)[rc/N-c-G-τ+B(N-1)z], thus there is an equilibrium z=N(c+G+τ)-rc/N(N-1)B for 0<N(c+G+τ)-rc/N(N-1)B<1. On the edge C-V, we have x+z=1 and ẋ=x(1-x)(P_C-P_V)=x(1-x)(G+τ)>0, thus the
direction of the dynamics goes from V to C. On the edge C-D, D can defeat C, as presented in Appendix A.1.




 §.§ C.2 The stabilities of equilibria


Theorem 6. In the conditions of Theorem 5, the fixed point (0, 1, 0) is stable, while the others ((0,0,1), (1,0,0), and (0, 1-N(c+G+τ)-rc/N(N-1)B, N(c+G+τ)-rc/N(N-1)B)) are unstable.

Proof.
(1) For (x,y,z)=(0,0,1), the Jacobian is

    J=[                G+τ                  0;                  0 -B(N-1)+c+G+τ-rc/N ],

thus the fixed point is unstable since G+τ>0.

(2) For (x,y,z)=(1,0,0), the Jacobian is

    J=[          -τ-G -(c+τ+G-rc/N);             0        c-rc/N ],

thus the fixed point is unstable since c-rc/N>0.

(3) For (x,y,z)=(0,1,0), the Jacobian is

    J=[     rc/N-c          0;     -(G+τ) rc/N-c-τ-G ],

thus the fixed point is stable since rc/N-c<0.

(4) For (x,y,z)=(0, 1-N(c+G+τ)-rc/N(N-1)B, N(c+G+τ)-rc/N(N-1)B), then the Jacobian is

    J=[               G+τ                 0; y(1-y)B(N-1)-yGyτ      y(1-y)(N-1)B ],

thus the fixed point is unstable since G+τ>0.



§ APPENDIX D




 §.§ D.1 Pool exclusion


In this subsection, we study the evolutionary dynamics of pool exclusion, which corresponds to the special case for pool-base switching strategy with T=0.
Then the replicator equations become

    {ẋ=x(P_C-P̅),
    ẏ=y(P_D-P̅),
    ż=z(P_F-P̅),
    .

where P_C, P_D, and P_F are the expected payoffs of cooperators, defectors, and pool excluders, respectively.

We also assume that exclusion never fails. In this condition, we give the expected payoffs as follows

    P_C   =   rc-c-(1-z)^N-1rc(N-1)y/N(1-z),  
    
    P_D   =   (1-z)^N-1rc/N(N-1)x/1-z,  
    
    P_F   =   rc-c-δ-τ.


Theorem 7. For r<N and δ+τ<rc-c, the system (<ref>) has four fixed points, namely, (x,y,z)=(0, 0, 1), (1, 0, 0), 
(0, 1, 0), and ([N(r-1)/r(N-1)]^1/N-1-(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
1-[N(r-1)/r(N-1)]^1/N-1).

Proof. By solving the system equations (<ref>), we can know that there are three vertex equilibrium points, namely, (0, 0, 1), (0, 1, 0), and (1, 0, 0).

Solving P_C=P_D results in
z=1-[N(r-1)/r(N-1)]^1/N-1.
Similarly, by solving P_C=P_F, we have
y=(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c. Thus when δ+τ<rc-c, there exists an interior fixed point ([N(r-1)/r(N-1)]^1/N-1-(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
1-[N(r-1)/r(N-1)]^1/N-1).

Then we investigate the dynamics on each edge of the simplex
S_3. On the edge C-D, we have z=0, resulting in
ẏ=y(1-y)(P_D-P_C)=y(1-y)(c-rc/N)>0. Thus the
direction of the dynamics goes from C to D.

On the edge D-F, we have
ż=z(1-z)(P_F-P_D)=z(1-z)(rc-c-δ-τ)>0, thus the
direction of the dynamics goes from D to EC.

On the edge C-F, we have
ẋ=x(1-x)(P_C-P_F)=x(1-x)(δ+τ)>0, thus the
direction of the dynamics goes from F to C.



 §.§ D.2 The stabilities of equilibria


Theorem 8. In the conditions of Theorem. 7, the three vertex equilibria are unstable, and the interior fixed point is neutrally stable surrounded by closed and periodic orbits.

Proof.
(1) For (x,y,z)=(0,0,1), the Jacobian is

    J(0,0,1)=[       δ+τ         0;         0 -rc+c+δ+τ ],

thus the fixed point is unstable since δ+τ>0.

(2) For (x,y,z)=(1,0,0), the Jacobian is

    J(1,0,0)=[          -δ-τ -(c+δ+τ-rc/N);             0        c-rc/N ],

thus the fixed point is unstable since 1-r/N>0.

(3) For (x,y,z)=(0,1,0), the Jacobian is

    J(0,1,0)=[         rc/N-c              0; -(rc/N-rc+δ+τ)       rc-c-δ-τ ],

thus the fixed point is unstable since rc-c-δ-τ>0.

(4) For
(x, y, z)=([N(r-1)/r(N-1)]^1/N-1-(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
(δ+τ)[N(r-1)/r(N-1)]^1/N-1/(r-1)c,
1-[N(r-1)/r(N-1)]^1/N-1), we define the equilibrium point as (x^**, y^**, z^**) hereafter, and the elements in the
Jacobian matrix are written as


    {∂f/∂x(x^**,y^**)=x^**[(1-x^**)∂/∂x(P_C-P_F)
    
    -y^**∂/∂x(P_D-P_F)],
    ∂f/∂y(x^**,y^**)=x^**[(1-x^**)∂/∂y(P_C-P_F)
    
    -y^**∂/∂y(P_D-P_F)],
    ∂g/∂x(x^**,y^**)=y^**[(1-y^**)∂/∂x(P_D-P_F)
    
    -x^**∂/∂x(P_C-P_F)],
    ∂g/∂y(x^**,y^**)=y^**[(1-y^**)∂/∂y(P_D-P_F)
    
    -x^**∂/∂y(P_C-P_F)],
    .


where


    {∂/∂x(P_C-P_F)   =-(x^**+y^**)^N-3rcy^**(N-1)(N-2)/N,
    ∂/∂y(P_C-P_F)   =-(x^**+y^**)^N-3rc(N-1)/N
       [(N-1)y^**+x^**],
    ∂/∂x(P_D-P_F)   =(x^**+y^**)^N-3rc(N-1)/N
       [(N-1)x^**+y^**],
    ∂/∂y(P_D-P_F)   =(x^**+y^**)^N-3rcx^**(N-1)(N-2)/N.
    .


Then we define that p_2=∂f/∂x(x^**,y^**)∂g/∂y(x^**,y^**)-∂f/∂y(x^**,y^**)∂g/∂x(x^**,y^**) and q_2=∂f/∂x(x^**,y^**)+∂g/∂y(x^**,y^**). Thus we know that


    p_2   =   x^**y^**(1-x^**-y^**)[∂/∂x(P_C-P_F)∂/∂y(P_D-P_F)
       -   ∂/∂y(P_C-P_F)∂/∂x(P_D-P_F)]
       =   x^**y^**(1-x^**-y^**)(x^**+y^**)^2N-4r^2c^2(N-1)^3/N^2
       >   0,


and


    q_2   =   x^**(1-x^**)∂/∂x(P_C-P_F)+y^**(1-y^**)∂/∂y(P_D-P_F)
       -   x^**y^**[∂/∂y(P_D-P_F)+∂/∂y(P_C-P_F)]
       =   x^**y^**(x^**+y^**)^N-3rc(N-1)(N-2)/N[(y^**-x^**)
       +   (1-y^**)-(1-x^**)]
       =   0.





We have q_2^2-4p_2<0 and q_2=0, therefore the eigenvalues of the Jacobian matrix are pure imaginary. The dynamics analysis of the interior of S_3 and the stability of interior fixed point can be found in the following subsection.




 §.§ D.3 The Hamiltonian system


To analyze the dynamics in the interior of S_3, we introduce a new variable ε=x/x+y, which represents the fraction of cooperators among members who do not contribute to the exclusion pool. This yields

    ε̇   =   xy/(x+y)^2(P_C-P_D)
       =   -ε(1-ε)(P_D-P_C).

By substituting x=ε(1-z) and P̅=x(P_C-P_D)+(1-z)(P_D-P_F)+P_F into ż=z(P_F-P̅), we have ż=z[x(P_D-P_C)-(1-z)(P_D-P_F)].
Thus we have


    {ε̇   =-ε(1-ε)[(1-z)^N-1rc(N-1)/N-rc+c],
    ż   =z(1-z)[rc-c-δ-τ-ε(rc-c)].
    .


By dividing the right-hand side of Eq. (<ref>) by the function ε(1-ε)z(1-z), we further have


    {ε̇   =-1/z(1-z)[(1-z)^N-1rc(N-1)/N-rc+c],
    ż   =1/ε(1-ε)[rc-c-δ-τ-ε(rc-c)].
    .


Let us introduce H(ε,z)=M(z)+L(ε), where M(z) and L(ε) are primitives of ż and ε̇, which are respectively given as


    M(z)   =   (-δ-τ)log(1-ε)-(rc-c-δ-τ)log(ε),
    
    L(ε)   =   (rc-c)[log(1-z)-log(z)]
       +   ∑_k=1^N-2N-2k(-1)^kz^k/k+log(z).


Then we obtain the Hamiltonian system as

    {ε̇=∂ H/∂ z,
    ż=∂ H/∂ε.
    .


Thus the system is conservative, and all constant level sets of H are closed curves.





§ ACKNOWLEDGMENTS



This research was supported by the National Natural Science
Foundation of China (Grant No. 61503062) and by the Slovenian Research Agency (Grant Nos. J1-7009 and P5-0027).

00

AxelrodR81
R. Axelrod,
Am. Political Sci. Rev. 75, 646-650 (1981).




Dur05 O. Durán and R. Mulet, Physica D 208, 257-265 (2005).


Tanimoto07 J. Tanimoto,  Phys. Rev. E 76, 041130 (2007).

Santos08 F. C. Santos, M. D. Santos and J. M. Pacheco, Nature 454, 213-216 (2008).

nowakm04 M. A. Nowak, A. Sasaki, C. Taylor and D. Fudenberg, Nature 428, 646 (2004).

zhang18 Y. Zhang, X. Chen, A. Liu and C. Sun, Appl. Math. Comput. 321, 641-653 (2018).

Tanimoto05 J. Tanimoto, Physica D  200, 1-24 (2005).


Wang11 W. X. Wang, Y. C. Lai, and D. Armbruster, Chaos  21, 033112 (2011).


chenxj08 X. Chen and L. Wang, Phys. Rev. E 77, 017103 (2008).

perc08 M. Perc and A. Szolnoki, Phys. Rev. E 77, 011904 (2008).


perc10 M. Perc and A. Szolnoki, Coevolutionary games-a mini review, BioSystems 99, 109-125 (2010).


perc17 M. Perc, J. J. Jordan, D. G. Rand, Z. Wang, S. Boccaletti, and A. Szolnoki, Phys. Rep. 687, 1-51 (2017).


nowakm06 M. A. Nowak, Science 314, 1560-1563 (2006).


fu08 F. Fu, C. Hauert, M. A. Nowak, and L. Wang, Phys. Rev. E 78, 026117 (2008).


ded04 D. J. De Quervain, U. Fischbacher, V. Treyer, and M. Schellhammer, Science 305, 1254 (2004).

chenx13 X. Chen, T. Gross, and U. Dieckmann, J. Theor. Biol. 335, 13-21 (2013).

szolnoki10 A. Szolnoki and M. Perc, EPL 92, 38003 (2010).

szolnoki12 A. Szolnoki and M. Perc, New J. Phys. 14, 093016 (2012).

perc_12_njp M. Perc and A. Szolnoki, New J. Phys. 14, 043013 (2012).

wang13 Z. Wang, A. Szolnoki, and M. Perc, Sci. Rep. 3, 1183 (2013).

chenx15 X. Chen, T. Sasaki, and M. Perc, Sci. Rep. 5, 17050 (2015).


Szolnoki11 A. Szolnoki, G. Szabó, and M. Perc, Phys. Rev. E 83, 036101 (2011).

chenx14 X. Chen, T. Sasaki, Å. Brännström, and U. Dieckmann, J. R. Soc. Interface 12, 20140935 (2014).

chenxj14 X. Chen, A. Szolnoki, and M. Perc, New J. Phys. 16, 083016 (2014).




rand09 D. G. Rand, A. Dreber, T. Ellingsen, D. Fudenberg, and M. A. Nowak, Science 325, 1272-1275 (2009).

gurerkq06 Q̈. Gürerk, B. Irlenbusch, and B. Rockenbach, Science 312, 108-111 (2006).

fehre02 E. Fehr and S. Gächter, Nature 415, 137-140 (2002).





wangz13 Z. Wang, C. Y. Xia, S. Meloni, C. S. Zhou, and Y. Moreno, Sci. Rep. 3, 3055 (2013).

yang2018 H. X. Yang and X. Chen, Appl. Math. Comput. 316, 460-466 (2018).

szolnoki17 A. Szolnoki and M. Perc, Phys. Rev. X 7, 041027 (2017).

perc15 M. Perc and A. Szolnoki, Sci. Rep. 5, 11027 (2015).

chenxj15 X. Chen, A. Szolnoki and M. Perc, Phys. Rev. E 92, 012819 (2015).

bodyr10 R. Boyd, H. Gintis, and S. Bowles, Science 328, 617-620 (2010).

sasaki15 T. Sasaki, S. Uchida, and X. Chen, Sci. Rep. 5, 8917 (2015).

Szolnoki16_njp A. Szolnoki and M. Perc, New J. Phys. 18, 083021 (2016).

sasaki13 T. Sasaki and S. Uchida, Proc. R. Soc. B 280, 20122498 (2013).

lik15 K. Li, R. Cong, T. Wu, and L. Wang, Phys. Rev. E 91, (2015) 042810.

szolnokia17 A. Szolnoki and X. Chen, Phys. Rev. E 95, 052316 (2017).

lik16 K. Li, R. Cong, and L. Wang, EPL 114, 58001 (2016).

liul17 L. Liu, X. Chen, and Szolnoki A. Sci. Rep. 7, 46634 (2017).

sui18 X. Sui, B. Wu, and L. Wang, EPL 121, 38003 (2018).

hellande98 E. Helland, Rev. Econom. Statist. 80, 141-153 (1998).

shimshackj05 J. P. Shimshack, and M. B. Ward, J. Environ. Econ. Manag. 50, 519-540 (2005).

liulj17 L. Liu, X. Chen, and S. Wang, Chinese Automation Congress (CAC), IEEE. 3961-3966 (2017).

sigmundk10 K. Sigmund, H. De Silva, A. Traulsen, and C. Hauert, Nature 466, 861-863 (2010).

schuster83 P. Schuster and K. Sigmund, J. Theor. Biol. 100, 533-538 (1983).

hofbauer03 J. Hofbauer and K. Sigmund, Evolutionary game dynamics, B. Am. Math. Soc. 40, 479-519 (2003).

Hauert2002
C. Hauert, M. S. De, J. Hofbauer, and K. Sigmund,
 J. Theor. Biol. 218, 187-194 (2002).

Sasaki2011
T. Sasaki and T. Unemi, J. Theor. Biol. 287, 109-114 (2011).

Tanimoto15 J. Tanimoto, Fundamentals of Evolutionary Game Theory and its Applications, (Tokyo, Japan, Springer, 2015).

wang2018 Q. Wang, N. He, and X. Chen, Appl. Math. Comput. 328, 162-170 (2018).

Rand11 R. H. Rand, M, Yazhbin, and D. G. Rand, Commun. Nonlinear Sci. Numer. Simul. 16, 3887-3895 (2011).


park18 J. Park, Appl. Math. Comput. 320, 425-436 (2018).

Zschaler10_njp G. Zschaler, A. Traulsen, and T. Gross, New J. Phys. 12, 093015 (2010).

reichenbach_06_pre T. Reichenbach, M. Mobilia and E. Frey, Phys. Rev. E 74, 051907 (2006).

Khalil96 H. K. Khalil, Noninear systems, (New Jersey, Prentice-Hall, 1996).

carr81 J. Carr, Applications of Center Manifold Theory, (New York, Springer Verlag, 1981).

Hassard81 B. D. Hassard, N. D. Kazarinoff, and Y. H. Wan,
 Theory and Applications of Hopf Bifurcation, (Cambridge, UK,
Cambridge University Press, 1981).


































