title: Statistical inference for principal components of spiked covariance matrices
authors: Zhigang Bao Xiucai Ding Jingming Wang Ke Wang
text: 
  
  
  
  
  
  
  Statistical inference for principal components
  
  
  ,
  ,
  ,
  
  
  Hong Kong University of Science and Technology t1 t3 t2
   Duke University t4 
  
  t1Z.G. Bao was  is partially supported by Hong Kong RGC grant  GRF 16301519 and NSFC 11871425.
  
  t3J.M. Wang  was partially supported by Hong Kong RGC grant ECS 26301517 and GRF 16300618.
  t2K. Wang was partially supported by Hong Kong RGC grant GRF 16301618 and GRF 16308219.
  Z.G. Bao, X.C.Ding, J.M. Wang, and K. Wang.
  
  
  
  Z.Bao
  J.Wang
  K.Wang
  
  Department of Mathematics,  
   Hong Kong University of Science and Technology, 
   Hong Kong
  
  e1
  
  E-mail:  *e3 
   
             *e2
  
  Department of Mathematics 
   Duke University 
  USA
  
  e4
  
  E-mail: 
  
  
  
  
  
  
  
  
  Zhigang Bao, Xiucai Ding, Jingming Wang, Ke Wang
  
  
  In this paper, we study the asymptotic behavior of the  extreme eigenvalues and  eigenvectors of  the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we derive the joint distribution of the extreme eigenvalues and the generalized components of the associated eigenvectors, i.e., the projections of the eigenvectors onto arbitrary given direction,  assuming that the dimension and sample size are comparably large. In general, the joint distribution is given in terms of linear combinations of finitely many  Gaussian and Chi-square variables, with parameters depending on the projection direction and the spikes. Our assumption on the spikes is fully general. First, the strengths of spikes are only required to be slightly above  the critical threshold  and   no upper bound on the strengths is needed. Second,  multiple  spikes,  i.e., spikes with the same strength, are allowed. Third,  no structural assumption is imposed on the spikes. 
  Thanks to the  general setting, we can  then apply the results to various high dimensional statistical hypothesis testing problems  involving both the eigenvalues and eigenvectors. Specifically, we propose accurate and powerful statistics to conduct hypothesis testing on the principal components. These statistics are data-dependent and  adaptive to the underlying true spikes.  Numerical simulations also  confirm the accuracy and powerfulness of our proposed statistics and illustrate significantly better performance compared to the existing methods in the literature. Especially, our methods are accurate and powerful even when either the spikes are small or the dimension is large.
  
  
  [class=MSC]
  [Primary ]60B20, 62G10
  [; secondary ]62H10, 15B52, 62H25
  
  
  
  random matrix, sample covariance matrix, eigenvector, spiked model, principal component, adaptive estimator
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  § INTRODUCTION
  
  
  Covariance matrices play  important role in multivariate analysis and high dimensional statistics, and find applications in many scientific fields.  Moreover,  many statistical methodologies and techniques rely on the knowledge of the structure of the covariance matrix, to name but a few, Principal Component Analysis, Discriminant Analysis and Cluster Analysis. For detailed discussions of the applications and methodologies, we refer the readers to the monographs <cit.> for a review. It is well-known in the high dimensional setting when the dimension is comparable with or much larger than the sample size, a direct application of the sample covariance matrix for hypothesis testing may result in untrustful conclusions <cit.>. Consequently, a thorough understanding of the distributions of the eigenvalues and eigenvectors of sample covariance matrices is in demand for high dimensional statistical inference.  
  
  In the literature of high dimensional statistics, a popular and sophisticated model is the spiked covariance matrix model proposed by Johnstone in <cit.>, where a finite number of spikes (i.e., eigenvalues detached from the bulk of the spectrum) are added to the spectrum of the population covariance matrix; see (<ref>) and (<ref>) below. Throughout the paper, with certain abuse of terminology, we use the word “spike" to represent either a detached eigenvalue 1+d_i (c.f. (<ref>), (<ref>)) or the whole rank one matrix corresponding to a detached eigenvalue (1+d_i)_i_i^* (c.f. (<ref>), (<ref>)). These spikes can have various practical meanings in different fields. For instance, they correspond to the first few important factors in  factor models arising from financial economics <cit.>, the number of patterns in  genetic variation across the globe <cit.>, the number of clusters in gene expression data <cit.> and the number of signals in single detection <cit.>. In this paper, we investigate the distributions of the principal components of the spiked sample covariance matrix, i.e, the sample counterparts of the extreme eigenvalues and eigenvectors (especially  those spikes) of the population covariance matrices. 
  The principal components of spiked sample covariance matrices play important roles in Principal Component Analysis for high dimensional data. A lot of work has been devoted to estimating the principal components in various settings.  For instance, sparse principal component analysis <cit.> is proposed to estimate the spiked eigenvalues and eigenvectors assuming some sparsity structure in the population eigenvectors; factor-model based estimators <cit.> for the eigenvectors are constructed if the population covariance matrix is of approximate factor-model type; and some regularization-based methods <cit.> have been proposed under various structural assumptions. 
  
  Despite the wide applications of the principal components in high dimensional statistics, most of the literature focus on the estimation part. Much less is known about their distributions, especially for the leading eigenvectors. As a consequence, a thorough study of the statistical inference for the population covariance matrix in the high dimensional setting is still missing, especially for hypothesis testing problems involving both eigenvalues and eigenvectors. For instance, eigenvectors and eigenspaces play an important role in statistical learning. However, the existing literature has only been able to test whether the eigenvectors or eigenspaces of the population covariance matrix are equal to some given ones under the assumption that the dimension is much smaller than the sample size <cit.>. For another example, in Principal Component Analysis, 
  the loadings are transformations of the original variables to the eigenvectors. They describe how much each variable contributes to a particular eigenvector and researchers are interested in hypothesis testing and constructing confidence intervals for them <cit.>. The loadings are scaled eigenvectors using their corresponding eigenvalues and therefore, the joint distribution of the extreme eigenvalues and eigenvectors of the sample covariance matrices will be needed to conduct the hypothesis testing on them.  
  
  Driven by these challenges, we study the joint distributions of the extreme eigenvalues and the generalized components of their associated eigenvectors for the spiked sample covariance matrices,  in the high dimensional setting. 
  Based on these results, we will be able to perform hypothesis testings with statistics constructed from both eigenvalues and eigenvectors. 
  
  
   
  
  Specifically, in this paper, we consider the sample covariance matrices of the form
  
      Q=TXX^*T^*,
  
  where T is a M× M deterministic matrix and X is a M× N random matrix with independent entries and 𝔼XX^*=I_M. Further, we assume that the population covariance matrix Σ:=TT^* admits the following form
  
      Σ=I_M+S,
  
  where S is a fixed-rank deterministic positive semi-definite matrix.  We first impose the following assumptions. 
  
  
  
    Throughout the paper, we suppose the following assumptions hold.
  
  
  (i)(On dimensionality ):  We assume that M≡ M(N) and N are comparable and there exist constants τ_2>τ_1>0 such that 
  
      y≡ y_N=M/N∈ (τ_1,τ_2).
  
  
  
  
  (ii)(On S): We assume that S admits the following spectral decomposition
  
      S=∑_i=1^r d_i _i_i^*,
  
  where r≥ 1 is a fixed integer. Here  d_1≥⋯≥ d_r >0 are the ordered eigenvalues of S, and _i=(v_i1,…, v_iM)^*'s are the associated unit eigenvectors. All d_i≡ d_i(N) may be N-dependent.
  
  
  (iii)(On X): For the matrix X=(x_ij), we assume that the entries x_ij≡ x_ij(N) are real random variables satisfying
  
      𝔼x_ij=0,    𝔼x_ij^2=1/N.
  
  Moreover, we assume the existence of  large moments, i.e., for any integer p≥3, there exists a constant C_p>0, such that
  
      𝔼| x_ij|^p≤ C_p<∞.
  
  We further assume that all √(N)x_ij's possess the same 3rd and 4th cumulants, which are denoted by κ_3 and κ_4 respectively. 
  
  
  
  Throughout the paper, for simplicity,  we will mainly work with the setting 
  
      T= Σ^1/2.
   
  We remark that our results hold for much more general T satisfying Σ=TT^*.  We refer to Remark <ref> for more discussions on the extension along this direction. 
  
  
   §.§ Summary of previous related theoretical results
  
  In this section, we summarize the results related to the spiked sample covariance matrix  from the Random Matrix Theory literature. 
  
  
  
  We denote by μ_1≥⋯≥μ_M ∧ N,  M ∧ N:=min{M,N},  the nontrivial eigenvalues of Q and  ξ_i the unit eigenvector associated with μ_i.  
  The primary interest of the sample covariance matrix Q lies in the asymptotic behavior of a few largest  μ_i's and the associated ξ_i's when N is large, under various assumptions of d_i's and 𝐯_i's. Significant progress has been made on this topic in the last few years.  It has been well-known since the seminal work of Baik, Ben Arous and Péché <cit.>  that the largest eigenvalues μ_i's undergo a phase transition (BBP transition) w.r.t. the size of d_i's. On the level  of the first order limit,  when d_i>√(y), the eigenvalue μ_i  jumps out of the support of the Marchenko-Pastur law (MP law) and converges to a limit determined by d_i, while  in the case of  d_i≤√(y), it sticks to the right end of the   Marchenko-Pastur (MP) law  (1+√(y))^2. In the former case, we call μ_i an outlier or outlying eigenvalue, while in the latter case we call μ_i a sticking eigenvalue. On the level of the second order fluctuation, it was revealed in <cit.> that a phase transition  for  μ_i takes place in the regime d_i-√(y)∼ N^-1/3. Specifically, if d_i-√(y)≪ N^-1/3 (subcritical regime), the eigenvalue μ_i still admits the Tracy-Widom type distribution; if d_i-√(y)≫ N^-1/3 (supercritical regime), the eigenvalue  μ_i is asymptotically Gaussian; while if d_i-√(y)∼ N^-1/3 (critical regime), the limiting distribution of the eigenvalue μ_i is some interpolation between  Tracy-Widom and Gaussian.  The works <cit.> and <cit.> are on real and complex spiked Gaussian covariance matrices respectively. On extreme eigenvalues, further study for more generally distributed  covariance matrices can be found in <cit.>. The limiting behavior of the extreme eigenvalues  has also been studied for various related models, such as  the finite-rank deformation of Wigner matrices <cit.>,  the signal-plus-noise model <cit.>, the general spiked β ensemble <cit.>, and also the finite-rank deformation of general unitary/orthogonal invariant matrices <cit.>.  
  
  In contrast, the study on the limiting behavior of the eigenvectors associated with the extreme eigenvalues is much less.   On the level of the first order limit, it is known that the ξ_i's  are delocalized and purely noisy in the subcritical regime, but has a bias on the direction of 𝐯_i in the supercritical regime.  We refer to <cit.> for more details of such a phenomenon. It was recently noticed in <cit.> that a d_i close to the critical point can cause a bias even for the non-outlier eigenvectors. On the level of the second order fluctuation, it was proved in <cit.> that the eigenvectors are asymptotically Gaussian in the subcritical regime, for the spiked covariance matrices. In the supercritical regime, a non-universality phenomenon was shown in <cit.> and <cit.> for the eigenvector distribution for the finite-rank deformation of Wigner matrices and the signal-plus-noise model, respectively. The non-universality phenomenon in the supercritical regime has been previously observed in <cit.> for the extreme eigenvalues of the  finite-rank deformation of Wigner matrices.  Here we also refer to <cit.> for related study on the extreme eigenstructures of various finite-rank deformed models from more statistical perspective.   
  
  
  
   §.§ An overview of our results
  
   In the theoretical part of this paper,  we will primarily  focus on the distribution of the eigenvectors ξ_i's associated with the outlying  eigenvalues μ_i's. That means, we will focus on the supercritical regime, in contrast to the work <cit.> where the eigenvector distribution in the subcritical regime was obtained. The results in the supercritical regime are particularly important for the statistical applications, since it is well-known that a reliable detection of spikes based on eigenvalues is only possible in this regime in general; see <cit.> for instance. Our assumption on the spikes is fully general (c.f. Assumption <ref>). Especially, we do allow d_i's to be divergent and   multiple (i.e. some d_i's are identical). In case that the spikes are simple (i.e. d_i's are all distinct), we also establish the joint distribution of the outlying eigenvalues and the associated  eigenvectors for the spiked covariance matrices.  This is the primary goal of the Principal Component Analysis from statistics point of view. More specifically, in this paper, we are interested in the  distribution of the largest μ_i's and the generalized component of the top eigenvectors, i.e., the projections of those eigenvectors onto a general direction. More specifically, let w∈ S^M-1_ℝ be any deterministic unit vector. We will study the limiting distribution of |⟨w,ξ_i⟩|^2 in the supercritical regime under general assumption of the spikes, and also state the joint distribution of |⟨w,ξ_i⟩|^2 and μ_i's in case that the spikes are simple.  
   We emphasize here that in case that a spike is multiple, one can also describe the joint distribution of eigenvalues and eigenvectors using the approach in this paper. But the result does not have a succinct form so we omit it from the statements of our main theorems; see Remark <ref> for more details. Nevertheless, we will describe (in certain equivalent form) and prove an extension of the joint eigenvalue-eigenvector distribution to the multiple case in  the application part, Section <ref>, and present applications of this result. 
   
  
  In the application part of this paper, we construct statistics to infer the principal components. We mainly focus on two hypothesis testing problems regarding the eigenspaces, (<ref>) and (<ref>). To our best knowledge, it is the first time that these problems are tackled for spiked covariance matrices  in the high dimensional regime (<ref>) without imposing any structural assumptions on the spikes.  Our proposed statistics make use of some plug-in estimators and are adaptive to the information of unknown spikes, for instance, their values and multiplicity. Thanks to the joint distribution of the eigenvalues and eigenvectors, we can easily establish the asymptotic distributions of our testing statistics; see Section <ref> for more details.
  Our methodology is simple, computationally cheap and easy to be implemented. Extensive numerical simulations lend strong support to our testing statistics. Especially, our proposed statistics are accurate and powerful regardless of the value of y and magnitude of the spikes. Moreover, for testing (<ref>), our statistics show better performance compared to the existing methods in the literature both in terms of accuracy and power. We point out that our methodology can be used to study other hypothesis testing problems regarding Principal Component Analysis and this will be discussed in Section <ref>.     
  
  
  
  
  
  
  
  
   Finally, we remark that our theory and applications are highly compatible. The theoretical results are not only interesting and natural on their own, they are also highly motivated by the  applications, which are all fundamental problems in the statistics literature. Especially, the generality of our assumptions in the theoretical part is not pursued only for technical interest; it is indeed indispensable in the application part. We leave more details about our contribution and novelties to Section <ref>,   since they can be better illustrated only after necessary notations and main results are stated.
  
  
   
  Notation:  Throughout the paper, the sample size N will be the fundamental parameter which goes to ∞.  The symbol o_N(·) stands for any quantity going to 0 as N goes to ∞. We use c and C to denote positive finite constants that do not depend on the  N. Their values may change from line to line.  For two positive quantities A_N, B_N depending on N we use the notation A_N ≍ B_N to denote C^-1A_N≤ B_N ≤ C A_N for some constant C>1. Further, we write A_N≐ B_N if A_N=B_N(1+o_N(1)). 
  
   For vectors v,w∈ℂ^N, we write v^*w=v,w for their scalar product. We emphasize here, unless otherwise specified,  the vectors in this paper are real vectors and thus v^*w=v^⊤w.   Further, for a matrix A, we denote by A_op its operator norm, while we use 𝐯 to represent  the ℓ^2 norm for a vector 𝐯.  
   
   We use double brackets to denote index sets, i.e. for n_1, n_2∈ℝ, n_1,n_2:= [n_1, n_2] ∩ℤ. In addition, we use 1_n=1/√(n)(1,…, 1)^* to denote the  
   n-dimensional normalized all-1 vector.  Further, we denote by 1(E) or 1_E the indicator function of an event E. 
  
  
  
  Organization: The paper is organized as the following: In Section <ref>, we state our main results and proof strategy. In Section <ref>, we discuss several applications of our results and present the simulation results.  In Section <ref>, we introduce some basic notions and preliminary results for later discussion. Section <ref> is devoted to the Green function representations of our eigenvalue and eigenvector statistics. Then in Section <ref>, we prove our main result, Theorem <ref>, based on a key technical recursive moment estimate for  some Green function statistics; see Proposition <ref>.  The proof of Theorem  <ref> is also stated in Section <ref>.  The proof of Proposition <ref> is  postponed to Appendix <ref>.  In addition, in Appendix <ref>, we collect some basic formulas concerning the derivatives of Green function, for the convenience of the reader. In Appendix <ref>, we provide the technical proofs for the statistics used in Section <ref>. Finally, Appendix <ref> collects some additional simulation results regarding non-Gaussian data.   
  
  
  
  § MAIN RESULTS, PROOF STRATEGY AND NOVELTIES
  
  In this section, we state our main results and explain our proof strategy and novelties. 
  We will need the following notions of a high-probability bound and convergence in distribution. The notion of stochastic domination was introduced in <cit.>, which provides a precise statement of the form “X_N is bounded by Y_N up to a small power of N with high probability”.
  (Stochastic domination)  Let
  
      X=(X_N(u): N∈ℕ, u∈ U_N),    Y=(Y_N(u): N∈ℕ, u∈ U_N)
  
   be two families of  random variables, where Y is nonnegative, and U_N is a possibly N-dependent parameter set.  We say that X is bounded by Y, uniformly in u, if for all small ϱ>0 and large ϕ>0, we have
   
      sup_u∈ U_Nℙ(|X_N(u)|>N^ϱY_N(u))≤ N^-ϕ
  
   for large N≥ N_0(ϱ, ϕ). Throughout the paper, we use the notation X=O_≺(Y) or X≺ Y when X is stochastically bounded by Y uniformly in u.   Note that in the special case when X and Y are deterministic, X≺ Y
  	means for any given ϱ>0,  |X_N(u)|≤ N^ϱY_N(u) uniformly in u, for all sufficiently large N≥ N_0(ϱ).
  
   
   In addition,  we also say that an N-dependent event ℰ≡ℰ(N) holds with high probability if, for any large φ>0,  
  
      ℙ(ℰ) ≥ 1- N^-φ,
  
  for sufficiently large N ≥ N_0( φ).
  
  
  
   
  Two sequences of random vectors, 𝖷_N ∈ℝ^k and 𝖸_N ∈ℝ^k, N≥ 1,  are asymptotically equal in distribution, denoted as 𝖷_N ≃𝖸_N, if they are tight and satisfy
  
      lim_N →∞( 𝔼f(𝖷_N)-𝔼f(𝖸_N) )=0
     
  for any bounded continuous function f:ℝ^k→ℝ. 
  
  
  
  
   §.§ Main results
  
  Since we will focus on the supercritical regime, we  further make the following assumption. First, we recall the notation 1, m:={1,⋯,m}.
    
  Let ϵ>0 be any small but fixed constant. Let d_i≡ d_i(N), i∈ 1, r be the eigenvalues of S in (<ref>). 
  There exists a maximum  integer r_0≡ r_0(ϵ)∈1, r, such that  for any i∈ 1, r_0,
  
      d_i- y^1/2 > N^-1/3 +ϵ
  
  for all sufficiently large N≥ N_0(ϵ). 
  Moreover,  for a fixed i∈ 1,  r_0, there exists a (unique) index set 𝖨≡𝖨(i) ⊂1, r_0 such  that i∈𝖨 and  for any t∈𝖨, 
  
      d_t= d_i,   δ_i:=min_j∈𝖨^c | d_i-d_j| > d_i^3/2(d_i-y^1/2)^-1/2N^-1/2+ ϵ,
  
  where we denote 𝖨^c:= 1,r∖𝖨. By definition, δ_t (or 𝖨(t)) is the same for all t∈𝖨(i). 
  
  
  
  
  
  
  It is known that the BBP phase transition takes place on the regime d_i-y^1/2∼ N^-1/3; see for instance <cit.>. Hence, (<ref>) ensures that we are in the supercritical regime. Further, note that we do not assume the spikes d_i≡ d_i(N) to be bounded in N. That means, we do allow d_i∼ N^c for any c>0, say. 
  In asd2, the first identity means that we allow  d_i to be multiple.  And the second inequality is the so-called  non-overlapping condition which guarantees that the distinct (possibly multiple) d_i's are well-separated such that the eigenvalues μ_i's corresponding to  distinct d_i's do not have essential overlap on the scale of fluctuation; see detailed explanation  in <cit.> for instance. Note that the prefactor d_i^3/2 is not included in the non-overlapping condition in <cit.>. But this factor is needed to cover the case when the N-dependent d_i is large. More precisely, when d_i is  large, as stated in Theorem 2.3 of  <cit.>, the fluctuation of  μ_i around its limiting location is of order O_≺(d_iN^-1/2), and thus  the lower bound on the RHS of the second inequality in asd2 which separates the limiting locations of  μ_i's corresponding to distinct d_i's is slightly larger than the fluctuation of  μ_i's. We emphasize here that in reality, it can certainly happen that two distinct d_i's are close enough to violate the non-overlapping condition. However, in this case, since the fluctuation of their sample counterparts, μ_i's, have essential overlap,  effective inference of d_i's based on μ_i's is believed to be impossible in general. 
  Also, since eigenvectors are sensitive to the eigenvalue gap, in this case, inference of _i's based on ξ_i's will also be unreliable. Therefore, the non-overlapping condition together with (<ref>) can be regarded as a nearly minimal condition for a reliable detection of spikes.  
  
  
  
  We will state our main results under Assumptions <ref> and <ref>, after necessary notations are introduced. Rewrite  the spectral decomposition of Σ as Σ= ∑_i=1^M σ_i 𝐯_i 𝐯_i^* = I_M + ∑_i=1^M d_i 𝐯_i 𝐯_i^*, where by Assumption <ref>, d_r+1=⋯=d_M=0. Further, we emphasize here that the specific choice of the orthonormal 𝐯_i's for i=r+1,…, M is irrelevant to our discussion since only ∑_j=r+1^M 𝐯_i𝐯_i^* will be involved. In the sequel, we fix an i and consider a (possibly) multiple d_i. Let 𝖨≡𝖨(i) be the index set of this multiple d_i in Assumption <ref>.  In order to study the generalized components of the eigenvectors of the |𝖨|-fold multiple  d_i, we introduce 
  
      Z_𝖨:=∑_t∈𝖨𝐯_t 𝐯_t^*,
   
   the orthogonal projection onto Span{𝐯_t}_t∈𝖨 
  and 
  the corresponding random projection 
  
      P_𝖨:=∑_t∈𝖨ξ_tξ_t^*,
    
  which is the sample counterpart of Z_𝖨. Note that in case |𝖨|>1, it is meaningless to do statistical inference for an individual  d_i𝐯_i𝐯_i^*, since in the multiple case, there is an arbitrariness in the  choice  of {𝐯_t}_t∈𝖨 as a basis for certain subspace. Hence, it is more natural to study Z_𝖨 and its sample counterpart. 
  For any unit 𝐰∈ℝ^M, denote its projection onto Span{𝐯_t }_t∈𝖨 by 
  
      _𝖨:=Z_𝖨,
  
  and its weighted projection onto Span{𝐯_j }_j∈ 1,M∖𝖨 by 
   
      ς_𝖨 := ∑_j∈ 1,M∖𝖨d_i√(d_j +1)/d_i -d_j⟨, _j ⟩_j
  
   with its normalized version 
   
      ς_𝖨^0: = ς_𝖨/‖ς_𝖨‖,    if ς_𝖨≠0;
      0,    otherwise.
  
  
    For any vectors 𝐚_l=(a_l(i))∈ℝ^M, l∈ℤ^+, we set the notations 
   
      𝐬_k_1,⋯, k_t(𝐚_1, ⋯, 𝐚_t) = 
        ∑_j=1^M a_1(j)^k_1⋯ a_t(j)^k_t.
  
   For instance, 𝐬_1,3(𝐚_1, 𝐚_2) = 
    ∑_j=1^M a_1(j)  a_2(j)^3. 
   
  For brevity, we introduce the following auxiliary functions for d>0
   
      𝚏(d):=y(1+d)/d(d+y)( 1+d(1+d)/d+y), 
          𝚐(d):= 2√((d+1)(d+√(y)))/d+y
      , 
         𝚑(d):=d+1/d+y ,    𝚕(d):=1+d/√(d(d+y)).
  
  With the above notations, we further define two symmetric matrices A_𝖨^ and B_𝖨^ of size (r+2)× (r+2), indexed by the vectors _𝖨, ς_𝖨, {_t}_t∈𝖨 and {_j}_j∈𝖨^c. 
  Here we recall the definition 𝖨^c= 1, r∖𝖨 from Assumption <ref>.  The only non-zero entries  of A_𝖨^ are given by the ones below and those followed by symmetry
  
  
      A_𝖨^(_𝖨, _𝖨) =2y 𝚑(d_i)^2(1+y 𝚑(d_i)^2) _𝖨^4,    A_𝖨^(ς_𝖨, ς_𝖨) = 𝚐(d_i)^2 _𝖨^2, 
          A_𝖨^(_t, _t) = 𝚑(d_i) ,
          A_𝖨^(_j, _j) = 𝚕(d_i)^2 _𝖨^2 , 
      
          A_𝖨^(ς_𝖨, _t) = 𝚐(d_i)√(𝚑(d_i))⟨_𝖨, _t ⟩,    A_𝖨^(ς_𝖨, _j) = 𝚐(d_i) 𝚕(d_i) ⟨ς_𝖨^0, _j ⟩_𝖨^2,  
          A_𝖨^(_t,_j)=√(𝚑(d_i))𝚕(d_i) ⟨_𝖨, _t ⟩⟨ς_𝖨^0, _j ⟩,   for  t∈𝖨, j∈𝖨^c.
  
  
  The only non-zero entries  of B_𝖨^ are given by the ones below and those followed by symmetry 
  
      B_𝖨^(_𝖨, _𝖨) =𝚏(d_i)^2 s_4(_𝖨),    B_𝖨^(ς_𝖨, ς_𝖨) = 𝚐(d_i)^2 s_2,2(ς_𝖨^0,_𝖨),  
         B_𝖨^(_t_1, _t_2) = 𝚑(d_i)s_1,1,2(_t_1,_t_2,ς_𝖨^0),    B_𝖨^(_j_1,_j_2)= 𝚕(d_i)^2 s_1,1,2(_j_1,_j_2, _𝖨),
         B_𝖨^(_𝖨,ς_𝖨 ) = 𝚏(d_i) 𝚐(d_i) s_1,3(ς_𝖨^0,_𝖨 ),    B_𝖨^(_𝖨, _t) = 𝚏(d_i) √(𝚑(d_i))s_1,1,2(_t, ς_𝖨^0, _𝖨),
         B_𝖨^(_𝖨, _j) = 𝚏(d_i) 𝚕(d_i) s_1,3( _j, _𝖨),    B_𝖨^(ς_𝖨, _t) = 𝚐(d_i)√(𝚑(d_i)) s_1,1,2(_t, _𝖨, ς_𝖨^0),
          B_𝖨^(ς_𝖨, _j) = 𝚐(d_i) 𝚕(d_i) s_1,1,2(_j, ς_𝖨^0,_𝖨), 
          B_𝖨^(_t, _j) = √(𝚑(d_i))𝚕(d_i) s_1,1,1,1(_t,_j,_𝖨, ς_𝖨^0),       for  t,t_1,t_2∈𝖨, j, j_1, j_2 ∈𝖨^c.
  
  
  
  To avoid the ambiguity on the realizations of the matrices A_𝖨^ and B_𝖨^ due to the possible permutations of the vector-indices _𝖨, ς_𝖨, {_t}_t∈𝖨, {_j}_j∈𝖨^c, we fix the ordering of these indices as follows:  _𝖨, ς_𝖨, followed by 
  _t's in  ascending order of t∈𝖨, and then followed by _j's in  ascending order of j∈𝖨^c. Then, A_𝖨^(_𝖨,ς_𝖨) will be the (1,2)-entry of  A_𝖨^, for instance. Correspondingly, in the sequel, we use the shorthand notation such as (α,β, {γ_t}_t∈𝖨, {δ_j}_j∈𝖨^c) to represent the vector with components α,β, γ_t, δ_j∈ℝ, t∈𝖨,j∈𝖨^c. Further, the ordering of the components in (α,β, {γ_t}_t∈𝖨, {δ_j}_j∈𝖨^c) is analogous to the ordering of the vector-indices of the matrices A_𝖨^ and B_𝖨^. Similar notations are used throughout the paper without further explanation. 
  
  
  
  With the above notations, we now state the first main theorem regarding the general components. 
  
  
  Suppose that Assumptions <ref>,  <ref>
   and the setting (<ref>) hold. Fix an  i∈ 1, r_0 and let ∈ S_ℝ^M-1 be any deterministic unit vector. In case d_i≡ d_i(N)→∞ as N→∞, we additionally assume that |y-1|≥τ_0 for some small but fixed τ_0>0. Then there exist random  variables Θ__𝖨^, Λ_ς_𝖨^, {Δ__t^}_t∈𝖨, {Π__j^}_j∈𝖨^c
  such that , P_𝖨 admits the following expansion
   
      , P_𝖨   =d_i^2-y/d_i(d_i+y), Z_𝖨+ 
      1/√(N(d_i^2-y)) Θ__𝖨^
      +ς_𝖨√(d_i- y^1/2)/ d_i Λ_ς_𝖨^
         
      +ς_𝖨^2 /Nd_i∑_t∈𝖨 (Δ__t^)^2-1 /N∑_j∈𝖨^cd_id_j/(d_i-d_j)^2 (Π__j^)^2 
         + O_≺(1/N^1/2+ε( ‖_𝖨‖^2/√(d_i^2-y) + ‖_𝖨‖‖ς_𝖨‖√( d_i- y^1/2)/d_i )) 
      
         
      +O_≺(1/N^1+ε(‖ς_𝖨‖^2/d_i + ‖_𝖨‖^2 ∑_j∈𝖨^cd_id_j/(d_i-d_j)^2 ))
  
  for some small constant ε>0, and 
  
      ( Θ__𝖨^, Λ_ς_𝖨^, {Δ__t^}_t∈𝖨, {Π__j^}_j∈𝖨^c) ≃𝒩(0, A_𝖨^ + κ_4  d_i^2-y/d_i^2 B_𝖨^).
   
   Here  𝒩(0, A_𝖨^ + κ_4  d_i^2-y/d_i^2 B_𝖨^) represents a Gaussian vector with mean 0 and covariance matrix A_𝖨^ + κ_4  d_i^2-y/d_i^2 B_𝖨^ with A_𝖨^ and B_𝖨^ defined in (<ref>) and (<ref>) respectively. 
   
   Here we further explain how to read off the information of  the limiting behaviour of , P_𝖨 from the expansion in (<ref>). The first term in the RHS of (<ref>) is the first order deterministic estimator of  , P_𝖨 which can be biased in the high-dimensional case, especially when d_i is a fixed constant independent of N. The second and the third terms in the RHS of (<ref>) are asymptotically normal, and the fourth  and fifth terms are (asymptotically) linear combinations of χ^2, according to (<ref>).  These four terms together describe the limiting distribution of , P_𝖨-d_i^2-y/d_i(d_i+y), Z_𝖨, after appropriate scaling. 
  We further emphasize here that the sizes of the first five terms may not be comparable and it is not uniformly determined which one is the  leading term in all cases. Under  different choices of d_i's and 𝐰, say, the leading term may change. However, in any case, the two error terms in (<ref>) are always smaller than the sum of the second to the fifth terms with high probability. This can be checked easily from the sizes of the entries in the covariance matrix A_𝖨^ + κ_4  d_i^2-y/d_i^2 B_𝖨^. 
  Hence, from the expansion  (<ref>), one can get the limiting distribution of , P_𝖨-d_i^2-y/d_i(d_i+y), Z_𝖨 in all cases. In the next  remark, we show how to get the limiting distribution for some specific examples. 
  
  
  
  
  
  
  
  
  
  
  
  
   If ∈Span{_t}_t∈𝖨, then _𝖨= and ς_𝖨 =0 (c.f. (<ref>), (<ref>)). Hence,  Λ_ς_𝖨^=0 and Δ__t^=0 for all t∈𝖨. The conclusion of Theorem <ref> is reduced to 
  
      , P_𝖨 =   d_i^2-y/d_i(d_i+y)+
      1/√(N(d_i^2- y))Θ_^
      -1 /N∑_j∈𝖨^cd_id_j/(d_i-d_j)^2 (Π__j^)^2
         +O_≺(N^-ε/√(N(d_i^2-y))+N^-ε/N∑_j∈𝖨^cd_id_j/(d_i-d_j)^2),
  
  for some small ε>0, and (Θ_^, {Π__j^}_j ∈𝖨^c) is asymptotically Gaussian with mean 0 and covariance matrix with entries given by the RHS of the following equations
  
      var(Θ_^) ≐ 2y 𝚑(d_i)^2(1+y 𝚑(d_i)^2) + κ_4  d_i^2-y/d_i^2𝚏(d_i)^2 s_4(w), 
         var(Π__j^) ≐𝚕(d_i)^2 + κ_4  d_i^2-y/d_i^2𝚕(d_i)^2 s_2,2 (_j, w), 
         cov(Θ_^,Π__j^) ≐κ_4  d_i^2-y/d_i^2𝚏(d_i) 𝚕(d_i) s_1,3(_j,),
         cov(Π__j^,Π__j̅^)≐κ_4  d_i^2-y/d_i^2𝚕(d_i)^2 s_1,1,2(_j, _j̅, ),
  
  for j, j̅∈𝖨^c. In particular, if κ_4=0, the limiting distribution of , P_𝖨 does not depend on the specific choice of ∈Span{_t}_t∈𝖨. Here we recall the notation A_N≐ B_N for A_N=B_N(1+o_N(1)). 
  
  
  If ∈Span{𝐯_j }_j∈ 1,M∖𝖨, then =0 and thus (<ref>) becomes
  
      , P_𝖨 = ς_𝖨^2/Nd_i∑_t∈𝖨 (Δ__t^)^2 + O_≺(‖ς_𝖨‖^2/N^1+εd_i).
  
  Hence,  the  eigenvectors ξ_i's are unbiased to any direction orthogonal to the spike directions _i's.  
  
  
  
  
   
  
    In this remark, we discuss several extensions. In (<ref>), we assumed T=Σ^1/2. But our result holds under much more general assumption on T. We can indeed extend our result to the matrix Q=TXX^*T^* with (M+k)× N random matrix X and M× (M+k) matrix T. As long as k∈ℕ is  fixed and TT^*=Σ satisfying (<ref>), our result remains true. Such an extension  has been discussed in Section 8 of <cit.> for the joint eigenvalue-eigenvector distribution in the subcritical regime and large deviation for eigenvector in the supercritical regime. It is easy to verify the validity of the same extension for the joint eigenvalue-eigenvector distribution in the supercritical regime.  The discussion in <cit.> relies on the rewriting  Q=TXX^*T^*=Σ^1/2YY^*Σ^1/2, where Y= (I_M 0)OX. Here 0 is the M× k zero matrix and O is some (M+k)× (M+k) orthogonal matrix.  It will be clear that all our arguments in the proof of Theorem <ref> are valid, as long as the isotropic local law of the matrix XX^* (c.f. Theorem <ref>) holds. So here it would be sufficient to have the isotropic local law for the matrix YY^*, which has been demonstrated in Theorem 8.1 of  <cit.>. Further,  we excluded the point 1 from the possible values of y≡ y_N in case d_i≡ d_i(N) diverges. This is essentially due to the same restriction in one technical result we need, Theorem <ref>, which is established in <cit.>.   We emphasize here that Theorem <ref> is only used in the case when d_i=d_i(N) grows with N, but not needed in the case when d_i is fixed. Therefore, when d_i is fixed, the result of Theorem  <ref> and also Theorem <ref> below also apply to all y≡ y_N∈ (τ_1,τ_2). Also, our result shall hold without the restriction |y-1|≥τ_0 even if d_i diverges. 
   But this extension will be left as future work.
   
   
     Our second result is  the joint eigenvalue-eigenvector distribution, i.e.,  joint distribution of the outlying eigenvalues and the generalized components of the associated eigenvectors. We state it   for the case when d_i is simple, i.e. 𝖨={i}. For simplicity, we abbreviate the notation 𝖠_{i} to 𝖠_i for 𝖠=,  P, Φ, etc. Further, we  use {i}^c to represent 1, r∖{i}. 
     
    Under the same assumptions as Theorem <ref>, with  𝖨={i} and  _i= , _i_i,  the conclusion of Theorem <ref> for the generalized component , P_i = |, _i|^2 holds.
      Additionally, there exists a random variable Φ_i such that the outlying eigenvalue admits the  expansion
  
      μ_i=1+d_i+y+y/d_i+√(d_i^2-y)/Φ_i+O_≺(√(d_i^2-y)/N^1/2+ε),
  
  for some small constant ε>0,  and 
  
      (Φ_i, Θ__i^, Λ_ς_i^, Δ__i^, {Π__j^}_j∈{i}^c) ≃𝒩(0, C_i^).
  
  Here  𝒩(0, C_i^) represents a Gaussian vector with mean 0 and covariance matrix C_i^ of size r+3. The lower right (r+2)× (r+2) corner of C_i^ is given by A_i^ + κ_4  d_i^2-y/d_i^2 B_i^ as in (<ref>) and (<ref>). The entries of the first row of C_i^ is given by the RHS of the following equations
  
      var(Φ_i) ≐ (1+ d_i^-1)^2(2+ κ_4 d_i^2-y/d_i^2s_4(_i)) ,
         cov(Φ_i, Θ__i^)≐ 2y 𝚑(d_i)^2 (1+d_i^-1)  , _i^2+ 
      κ_4 d_i^2-y/d_i^2 (1+d_i^-1) 𝚏(d_i) s_2,2(_i, _i),
         cov(Φ_i, Λ_ς_i^) ≐κ_4 d_i^2-y/d_i^2𝚐(d_i)(1+d_i^-1) s_1,1,2(ς_i^0, _i, _i), 
         cov(Φ_i, Δ__i^)≐κ_4 d_i^2-y/d_i^2√(𝚑(d_i))(1+d_i^-1) s_1,3(ς_i^0, _i) ,
         cov(Φ_i, Π__j^)≐κ_4 d_i^2-y/d_i^2𝚕(d_i)(1+d_i^-1) s_1,1,2(v_j, _i, _i),   for  j ∈{i}^c.
  
  Here we recall the notation A_N≐ B_N for A_N=B_N(1+o_N(1)).
   
  
  
  
  
   
    Here we remark that in the supercritical regime, a generalized CLT for the eigenvalues  has been established in <cit.> previously, for fixed d_i's which are away from y^1/2 by a constant order distance. When there is a multiple d_i in the supercritical regime with multiplicity |𝖨|, it is known from <cit.> that the corresponding eigenvalues {μ_t}_t∈𝖨 will converge jointly to the eigenvalues of a |𝖨|× |𝖨| Gaussian matrix  GOE. Since it is not convenient to express the distribution of the eigenvalues of this fixed-dimensional GOE and their dependence with the the generalized components of ξ_i's, we are not going to state the joint eigenvalue-eigenvector distribution in the multiple case here. Nevertheless,   we will state the joint distribution of the generalized components of  ξ_i and all the matrix entries of this limiting GOE in Appendix <ref> which equivalently describes the joint eigenvalue-eigenvector distribution; see Proposition <ref>.
   
  
  
  
  
  
  
  
  
  
  
   §.§ Proof strategy and novelties
   
  
  Finally, here we remark that a related problem has been previously  studied in <cit.> for the so-called matrix denoising model, where the distribution of the leading singular vectors of this model was studied. Due to the additive structure of this model, the distribution of the singular vector may depend on the structure of the deformation and the entire distribution of entries of the noise matrix (rather than their first 4 moments only), and  may not be Gaussian or Chi-square or linear combinations of them. 
  Such a phenomenon is called non-universality, which exists in  the additive models <cit.>. However, such a phenomenon does not show up for the spiked covariance matrix, as one can see from Theorem <ref>, where the distribution has a Gaussian nature in the sense that it is a polynomial of Gaussian variables. This is essentially due to the multiplicative structure of the spiked covariance matrix where the structure of the spikes are smoothed out by the random matrix X. In addition, we emphasize here that in <cit.> the assumption of the strengths of the deformation, counterpart of d_i's,  is much more limited than the assumption here. In <cit.>, the strengths are assumed to be bounded and thus cannot grow with N, and also the strengths are away from the critical threshold by a constant order distance. Further, the strengths in  <cit.> are assumed to be simple, and thus no multiplicity is allowed, and also distinct strengths are away from each other by a constant order. In addition, only the projection of the random singular vector onto the directions of the deformations are discussed therein. Finally, no joint distribution of eigenvalue and eigenvectors is obtained in <cit.>. 
  The generality of the settings and the results bring various technical problems but meanwhile they are highly motivated by the applications. In the sequel, we highlight several novelties from both theoretical and application point of views. 
  
  Similarly to <cit.>, we start with  Green function representations of the eigenvector and eigenvalue statistics. It turns out that both the eigenvector and eigenvalue statistics can be expressed (approximately)
  in terms of some quadratic forms of the Green function of the matrix XX^*. Then a recursive moment estimate for the linear combinations of these quadratic forms leads to the joint distribution of the eigenvector and eigenvalues statistics. A key technical input is the isotropic local laws derived in <cit.>. Equipped with this general strategy, we face several new technical obstacles, in contrast to the discussions in <cit.>.  First, since the d_i's could be either very close to the critical threshold or diverging, and meanwhile could be equal or close to each other, 
  the control of the sizes of the terms (especially the error terms) becomes much more delicate. One needs to keep tracking the dependence of the size of terms on d_i-√(y), d_i-d_j and d_i carefully to conduct a unified analysis in all cases of d_i's. Especially, in case that d_i is diverging, one needs to exploit a hidden cancellation between two quadratic forms of the Green function, which is absent in case that d_i is fixed. In order to see such a cancellation, one needs to adopt the recently established nearly optimal convergence rate of the so-called eigenvector empirical spectral distribution (VESD) in <cit.>. Second, in contrast to <cit.>, where only the projection to the direction of the deformation is considered, here we consider the projection to arbitrary direction. Especially, when one considers the projection to the orthogonal complement 
  of the direction of the deformation, the size of the whole projection will degenerate to a smaller order. In order 
  to study the fluctuation of the eigenvector projection onto arbitrary directions, including the direction orthogonal to the one of the spike, one needs to express the eigenvector projection in terms of the Green function up to a higher order term, and involve the higher order term in the recursive moment estimate, since it could be significant.  Third, the joint distribution of the eigenvalue and eigenvector statistics is obtained for the first time in the supercritical regime for the whole range of d_i. Such a complete result was even not available for the eigenvalue statistics only. 
  
  All the theoretical novelties are well motivated by our applications. First, in most of mathematical work on spiked models, d_i is assumed to be bounded. However, in many  popular statistical models such as
   the factor model <cit.>, 
  d_i≡ d_i(N) could be diverging. We provide a unified result in the whole supercritical regime, no matter d_i is close to the threshold or diverging. Practically, that means our results can be applied no matter the spike is weak or strong.   
  Second, in the application part, we consider two hypothesis testing problems. The first is to test whether an eigenspace formed by any part of the spikes is equal to some given subspace, while the second is to test whether it is orthogonal to certain given subspace. Both questions are significant in the statistics literature. We raised two testing statistics for these two problems respectively, and the limiting distribution of the first statistic relies on our joint eigenvalue-eigenvector distribution in case ∈Span{_t}_t∈𝖨, while  the limiting distribution of the second statistic relies on the joint eigenvalue-eigenvector distribution in case ∈Span{𝐯_j }_j∈ 1,M∖𝖨. This explains the necessity for us to derive the distribution of the projection onto general directions. Third,  in two applications, if we construct the testing statistics, using the result in 
  Theorem <ref> solely, the limiting distribution of the statistics will contain the parameter d_i, which is normally unknown in real application. Hence, in order to construct  adaptive statistics which do not depend on the unknown parameter    d_i, we use a plug-in estimator of d_i which is given in terms of μ_i. Then, in order to derive the distribution of these adaptive  statistics, we have to establish the joint eigenvalue-eigenvector distribution, as what we have in Theorem <ref>, and its multiple extension in Proposition <ref>. 
  
  
  
  
  
  
  
  
  
  
  § STATISTICAL INFERENCE FOR PRINCIPAL COMPONENTS
   
  In this section, we apply our results and some of their variants to the statistical applications regarding the inference of principal components. We focus our discussion on the hypothesis testing regarding the eigenspaces of covariance matrices.  Eigenspaces of covariance matrices are important in many statistical methodologies and computational algorithms. A lot of efforts have been made to infer the eigenspace of the covariance matrices in the setting M ≪ N, for instance, see <cit.>. 
  
  In this section, we consider a generic index set ℐ⊂ 1, r_0, which may contain indices for both simple and multiple d_t's.  Further, we set 
   
      Z_ℐ= ∑_t∈ℐv_tv_t^*.
  
   We remark here that Z_ℐ  shall be regarded as an extension of Z_𝖨(i)  defined in (<ref>), in the sense that the former may be constituted of v_t's 
   associated with distinct d_t's. 
  
  Specifically,  in the literature, researchers are particularly interested in testing the following hypothesis: for ℐ⊂1, r_0,
  
      𝐇_0^(1):  Z_ℐ=Z_0  vs 𝐇_a^(1): Z_ℐ≠ Z_0
     
  for a given projection Z_0. For the general alternatives 𝐇_a^(1), we are particular interested in testing whether Z_0 is in the complement of Z_ℐ. Specifically, the hypothesis testing problem can be formulated as 
  
      𝐇_0^(2): Z_ℐ⊥ Z_0  vs 𝐇_a^(2): Z_ℐ⊥̸Z_0.
   
  Note that (<ref>) is the complement of the test considered in <cit.> and hence it can be used to study the alternative in <cit.>.  
  
   In Section <ref>, we propose accurate and powerful statistics for the aforementioned two hypothesis testing problems (<ref>) and (<ref>) in the high dimensional regime (<ref>).  We construct test statistics using some plug-in estimators and then derive their distributions utilizing the joint distribution of the eigenvalues and eigenvectors established in Section <ref> and its extensions in Appendix <ref>. For (<ref>) and (<ref>), the plug-in estimators are nonlinear shrinkers of the sample eigenvalues. Consequently, the proposed statistics are adaptive to d_i's.  We mention that this methodology can be potentially applied to perform statistical inference  and build up confidence intervals for other important statistics related to the principal components. For instance, the loadings of principal components <cit.>, the shrinkage of eigenvalues <cit.>, the number of spikes <cit.>, the estimation of eigenvectors <cit.> and the invariant estimator for covariance matrices <cit.>. These applications will be studied  in the future.         
  
  
  
   
  
  
  
  
  
  
  
   §.§ Test statistics and their asymptotic distributions
   In this section, we propose statistics to test (<ref>) and (<ref>). We start with  (<ref>). 
  In what follows, we construct a data-dependent statistic to address the high dimensional issue. Denote Z_0= ∑_i∈ℐu_iu_i^*.  
  We  first study 
  
      𝒯:=∑_i ∈ℐ(⟨u_i,   P_ℐu_i⟩ - ϑ(d_i)),
  
  where 
  
      P_ℐ=∑_i ∈ℐξ_i ξ_i^* ,   ϑ(d)= d^2-y/d(d+y)
  
  and d_i=γ(μ_i) is a nonlinear shrinkage of the sample eigenvalues  denoted as
  
      d_i=γ(μ_i), γ(x)=1/2(-y+x-1)+1/2√((-y+x-1)^2-4y).
  
   We remark here that P_ℐ  shall be regarded as an extension of P_𝖨(i)  defined in (<ref>), in the sense that the former may be constituted of ξ_t's 
   associated with distinct d_t's.  Further, we remark here, according to the definition  in (<ref>), the statistic 𝒯 does not depend on the specific choice of the basis {u_i} of Z_0. Hence, we have a freedom to choose any basis {u_i} of Z_0 in the sequel. 
  
  
  Since we are studying general ℐ in this section, the indices in ℐ may not belong to the same multiple d_t's. To facilitate our discussion in the sequel, we do a decomposition of  ℐ into subsets with each consisting of the indices for one multiple (or simple) d_t. 
   For ℐ= {i_1, ⋯, i_r_*}⊂ 1, r_0 , we assume that ℐ=⋃_k=1^ℓℐ_k  for some fixed integer ℓ such that ℐ_k∩ℐ_j=∅ for k ≠ j ∈ 1, ℓ. We assume that (<ref>) holds for all the d_i, i∈ℐ. For each i∈ℐ, there is a k_i∈ 1, ℓ, such that i∈ℐ_k_i. 
   Moreover, we suppose that for 1 ≤ k ≤ℓ, d_t, t∈ℐ_k are all the same,  and d_i≠ d_j for i ∈ℐ_k_i, j ∉ℐ_k_i. Note that by definition ℐ_k_i≡𝖨(i) (c.f.  Assumption <ref>).  Further note that ℓ=r_* corresponds to the case that all the spikes in ℐ are simple , ℓ=1 corresponds to the case that all the spikes are equal and 1< ℓ <r_* corresponds to a mixture case.
  
   
   
   For the brevity of the discussion in the first test problem (<ref>), we further restrict ourselves to the case satisfying the following assumption. 
     Let the index set ℐ⊂ 1, r_0 be defined above. We assume that for any i∈ℐ, the following inequality holds
   
      1/N∑_j∈ (𝖨(i))^cd_id_j/(d_i-d_j)^2≤  N^-ε1/√(N)(d_i^2-y).
   
   for some small but fixed ε>0. Here i∈ℐ_k_i≡𝖨(i) for some k_i∈ 1, ℓ.
   
    
   We remark here that the inequality (<ref>) insures that the χ^2 terms in (<ref>) are suppressed by the Gaussian term. We impose such a condition in order to simplify the discussion in the application part, thanks to the simplicity of the Gaussianity. But our result can be applied without this additional assumption. In the general case, we need to work with a linear combination of Gaussian and χ^2 variables. For brevity, we omit such a general discussion and leave it to the future work.  
    
  
   
     We record the results regarding the asymptotic distribution of (<ref>) in the following theorem and postpone its proof to Appendix <ref>.   
  
  
   Suppose that Assumptions <ref> ,  <ref>,  and the setting (<ref>) hold.  In case d_i≡ d_i(N)→∞ as N→∞ for some i∈ℐ, we additionally assume that |y-1|≥τ_0 for some small but fixed τ_0>0. Suppose that 𝐇_0^(1) of (<ref>) and Assumption <ref> hold. 
  For the statistic  (<ref>), we have that 
  
      √(N)𝒯/√(𝚅_1(𝐝_ℐ))≃𝒩(0,1),
  
  where 𝚅_1(d_ℐ), d_ℐ=(d_i_1, ⋯, d_i_r_*) is defined as 
  
      𝚅_1(d_ℐ):= α^* C_ℐα.
  
  Here α=(α_1,⋯, α_2 r_*)^* ∈ℝ^2r_* is defined as
  
      α_k= {[ -y(d_i_k^2+2d_i_k+y)(d_i_k+y)^-2(d_i_k^2-y)^-1/2,                                      1 ≤ k ≤ r_*;;                             (d_i_k-r_*^2-y)^-1/2,                                r_*+1 ≤ k ≤ 2 r_*, ].
  
  and C_ℐ is a positive definite matrix of dimension 2r_* and explicitly defined in Proposition <ref>. Particularly,  when all the spikes d_t, t ∈ℐ are equal to d_e,  i.e., ℐ=𝖨(i) for some i, we have that  
  
  
  
      𝚅_1( d_ℐ)=
      2(d_e^2-y)^-1( y𝚑(d_e)^2 |ℐ| - y(d_e^2+2d_e+y)(1+d_e)/(d_e+y)^2d_e)^2 
                 + 2(d_e^2-y)^-1(y𝚑(d_e)^2|ℐ|  + y^2𝚑(d_e)^4(|ℐ|- |ℐ| ^2)  ) 
                + κ_4 ( 𝚏(d_e)/d_e -  y(d_e^2+2d_e+y)(1+d_e)/(d_e+y)^2d_e^2)^2 ∑_k, t∈ℐs_2,2(_k,_t),
  
  where 𝚑(·) and 𝚏(·) are defined in (<ref>) and (<ref>).
  
  
  
  
  
  
  
  
  By Theorems <ref> and <ref>, we can construct a pivotal statistic. Denote 
  
      𝕋_1=√(N)𝒯/√(_1(d_ℐ)),  d_ℐ=(γ(μ_i_1), ⋯, γ(μ_i_r_*)).
  
  We mention that  (<ref>) is  adaptive to the d_i's by utilizing their estimators (<ref>). We summarize the distribution of 𝕋_1 in the corollary below, whose proof will also be postponed to Appendix <ref>. 
  
  
  
  Under the assumptions of Theorem <ref>, we have that 
  
      𝕋_1≃𝒩(0,1).
  
  
  
  
  Since 𝕋_1 is asymptotically pivotal, we will use (<ref>) as our statistic for the testing of (<ref>). For an illustration, we record the behavior of our statistic for a single  spike model (i.e., r_0=r_*=1) in Figure <ref>. The more general and extensive simulations will be conducted in Section <ref>. We find that under the null hypothesis of (<ref>), our proposed statistic is close to 𝒩(0,1) for different values of d and hence it is suitable for the hypothesis testing problem (<ref>).  We mention that even though we have not justified the case d_i diverges under the assumption y=1 theoretically, our statistic is still accurate and powerful according to empirical illustrations for this case.  Hence, in the sequel, we also present the simulation results for the case y=1 for more extensive simulation study. 
   
                  
  
  
  
  
  
  
  
  
  In what follows, we provide a few  examples with explicit formulas of  _1(·).  These will be used for the simulations in Section <ref>.  
  
  We consider that ℐ={1,2}, both d_1 and d_2 are simple and v_i=e_i, i=1,2. In this case, we have that 
  
      𝚅_1(d_1, d_2)= ∑_i=1^2 α^*
       [
       [ A_11(d_i) A_12(d_i); A_21(d_i) A_22(d_i) ]]
       α,
  
   where
   
      α  = (  -y(d_i^2+2d_i+y)/(d_i+y)^2(d_i^2-y)^1/2,  (d_i^2-y)^-1/2) ^* 
          A_11(d_i) = 2(1+d_i^-1 )^2 +  κ_4 (1-yd_i^-2)(1+d_i^-1)^2,
      
          A_12(d_i) = 2y 𝚑(d_i)^2 (1+d_i^-1 ) + κ_4 (1-yd_i^-2) 𝚏(d_i)(1+d_i^-1),
          A_22(d_i) = 2y 𝚑(d_i)^2 (1+ y 𝚑(d_i) ^2 ) + κ_4 (1-yd_i^-2) 𝚏(d_i)^2,
  
  and  the functions 𝚑(·) and 𝚏(·) are defined in (<ref>) and (<ref>).
  
  
  
  We consider the case that ℐ={1,2} with d_1=d_2=d and v_i=e_i, i=1,2. In this case, we have that 
  
      𝚅_1(d, d)=   2(d^2-y)^-1( 2y𝚑(d)^2 - y(d^2+2d+y)(1+d)/(d+y)^2d)^2 
                 + 2(d^2-y)^-1(2 y𝚑(d)^2  -2 y^2𝚑(d)^4  ) 
                + κ_4 ( 𝚏(d)/d -  y(d^2+2d+y)(1+d)/(d+y)^2d^2)^2 ∑_k, t∈ℐs_2,2(_k,_t).
  
  
  
   
  
  Next, we consider the hypothesis testing (<ref>). In this case, we further assume that the true model or the population matrix Σ only contains supercritical spikes, i.e., 
  
      r_0=r.
  
  It will be seen that if there exist  subcritical spikes, one will  need to provide a plug-in estimator of the subcritical d_i's in order to  raise a testing statistic which is adaptive to all the spiked eigenvalues.
  However, it is well-known now that an effective detection of subcritical d_i's  based on μ_i's is impossible in general, unless one knows additional information such as the  structure of v_i's.  
   And also, indeed,  in many applications, d_i's are very large and even divergent, and thus are certainly supercritical. Hence, in the sequel, we will focus on the case when (<ref>) is satisfied. 
    Suppose that in this case Z_0= ∑_j∈𝒥u_ju_j^* for some fixed index set 𝒥 and {_j}_j∈𝒥 is a family of orthonormal vectors.
  
  
  We employ that 
  
      𝕋_2=∑_i ∈ℐ, j∈𝒥⟨ξ_i, u_j ⟩^2.
  
  
  
  
  
  
  The asymptotic distribution of 𝕋_2 is recorded in the following theorem. It turns out that its asymptotic distribution coincides with linear combinations of χ^2 variables. 
   For convenience, we  first define d:= (d_1, …, d_r) and 
  
      𝚚(d):=  max_i∈ℐ, j ∈𝒥∑_k∈ 1,M∖ℐ𝚑(d_i)d_i(d_k+1)/(d_i-d_k)^2⟨_j,_k⟩^2
  
    which depends on the subspace Z_0 and all the  d_i's for i∈ 1, r. Here d_r+1=⋯ d_M=0. We emphasize that  all  d_i's for i∈ 1, r satisfy  asd and asd2 in this part.
  
  
  
   Suppose that Assumptions <ref> ,  <ref>,  and the settings (<ref>) and (<ref>) hold.  In case d_i≡ d_i(N)→∞ as N→∞ for some i∈ℐ, we additionally assume that |y-1|≥τ_0 for some small but fixed τ_0>0.
  Suppose that H_0^(2) of (<ref>) holds true.
  For the statistic 𝕋_2 defined in eq_finalstat, we have 
  
      N 𝕋_2/𝚚(d)≃g^*Ug/𝚚(d),
  
  where g∈ℝ^|ℐ||𝒥|, g∼𝒩(0,I_|ℐ||𝒥|), and U≡U(d) is a symmetric matrix of dimension |ℐ||𝒥| defined explicitly in Proposition <ref>.
  
  
  
  We remark here that in asymp.test.ortho, we add the factor 1/𝚚(d) on both sides to scale them to order one random variables.   Since we use the notation “ ≃ " which is defined in Definition <ref> , we have to require the sequence of variables on both sides to be tight.
  
  
  
  
  
  
  
  
  
  
  The results of Theorem <ref>, especially 𝚚(d) and 𝐔, still contain the values of d_i, i ∈ℐ and also the other nonzero spikes d_j, j∈ℐ^c which are all supercritical (c.f. (<ref>)) so that  we can  use   (<ref>) to estimate them all.
  To construct a data-dependent statistic, we can use the plug-in estimator (<ref>)  to generate critical values of the hypothesis testing (<ref>) using the samples. Recall d= (d_1, …,d_r).
  
   Under the assumptions of Theorem <ref>, we have that
  
      N𝕋_2/𝚚(d )≃g^*Ug/𝚚(d ),
  
  where U = U (d ). 
  
  
  
  We can use our statistic (<ref>)  with the critical values generated from Corollary <ref> to study the hypothesis testing problem (<ref>). 
   Here we shall point out that although our statistic  N𝕋_2/𝚚(d ) is adaptive to the d_i's, it is nevertheless dependent of {v_i}_i∈ 1, r∖ℐ and also  a κ_4 term which involves some {v_i}_i∈ℐ-dependent parameters of the form s_1,1,1,1(·, ·, ·,·); see the definition of U in Proposition <ref>. Hence, first of all, we shall only apply our statistic N𝕋_2/𝚚(d ) in case either {v_i}_i∈ 1, r∖ℐ is known a priori, or ℐ= 1, r such that the set {v_i}_i∈ 1, r∖ℐ is empty. In practice, this restriction is mild and  fits the following real scenario: if some of the v_i's are already known, we only need to do inference for those unknown v_i's, while if none of the v_i is known a priori, we consider the inference for all {v_i}_i∈ 1, r together. Certainly, it is also natural to consider a part of v_i's even if none of them is known, as  what we did in the test (<ref>). Nevertheless, due to the restriction of the theoretical result, we focus on the aforementioned scenario, which is more restricted but still very natural.  Second, the unknown κ_4 term will be absent in case we consider the Gaussian matrix X which is often the case in reality. Hence, in Gaussian case, we can apply our statistic directly if we restrict ourself to the aforementioned scenario of v_i's. Nevertheless, for the reader's reference,  we also present our simulation study in the Appendix for the two-point case, as if the additional parameter, the κ_4 term, was known a priori.  Here, we remark that the restriction of scenario to apply our theoretical result for the test (<ref>), which is not necessary for (<ref>), is actually quite reasonable. Note that, in (<ref>), the necessary parameters from {v_i}_i∈ℐ are completely fixed by the  null hypothesis, and the distribution of our statistic (under the null hypothesis) can be expressed in terms of these given parameters. However, in the test (<ref>), our null hypothesis is Z_ℐ⊥ Z_0. In case that the rank of the projection of the given Z_0 is small, as a low-rank subspace living in the complement of Z_0, Z_ℐ can have many choices and thus there is a big uncertainty on the unknown parameters of Z_ℐ  which cannot be fixed by Z_0. 
  
  
  
  For an illustration, we record the behavior and power of our statistic for a single spiked model (i.e., r_0=r=1) in Figure <ref>.
  We find that our proposed statistic is close to Chi-square distribution with one degree of freedom for various values of d and hence it can be applied for testing (<ref>)  for this single spike model. For more general case, the asymptotic distribution of (<ref>) is a linear combination of Chi-square distributions. We will conduct extensive simulations in Section <ref>. 
  
  
  
  
  
  
  
  
  
  We next consider a few examples to specify the asymptotic distribution stated in Theorem <ref> and the results will be used in Section <ref>. 
  
  We consider that r_0=3,  ℐ={1,2}, d_i, i=1,2,3 are simple and satisfy asd, asd2 and v_i=e_i, i=1,2,3 and Z_0=e_3 e_3^*+e_4 e_4^*. In this case, since  ς_{1}^e_3=d_1√(d_3+1)/d_1-d_3e_3, ς_{2}^e_3=d_2√(d_3+1)/d_2-d_3e_3  and ς_{1}^e_4=ς_{2}^e_4=e_4 and s_1,1,1,1(e_i_1, e_i_2,  e_j_1, e_j_2)=0 for i_1,2= 1,2, j_1,2= 3,4. Further, 𝚚(d)= 𝚚(d_1,d_2, d_3)= max{(d_3+1) d_1𝚑(d_1)/(d_1-d_3)^2, 𝚑(d_1)/d_1, (d_3+1) d_2𝚑(d_2)/(d_2-d_3)^2, 𝚑(d_2)/d_2}. We have that the statistic N 𝕋_2/𝚚(d) will be asymptotically distributed as 
   
  
      1/𝚚(d) g^* diag( (d_3+1) d_1𝚑(d_1)/(d_1-d_3)^2, 𝚑(d_1)/d_1, (d_3+1) d_2𝚑(d_2)/(d_2-d_3)^2, 𝚑(d_2)/d_2) g,
  
  
  where g∈ℝ^4 is a Gaussian random vector such that g∼𝒩(0, I_4).  
  
  
  
  
   We consider that r_0=3, ℐ={1,2}, d_1=d_2=d, d_3 is distinct from d by a distance of order 1, and 𝐯_i=e_i, i=1,2,3. In this case, 𝚚(d)= max{(d_3+1) d𝚑(d)/(d-d_3)^2, 𝚑(d)/d}. We have that the statistic N𝕋_2/𝚚(d) will be asymptotically distributed as 
  
      1/𝚚(d) g^* diag( (d_3+1) d 𝚑(d)/(d-d_3)^2, 𝚑(d)/d, (d_3+1) d 𝚑(d)/(d-d_3)^2, 𝚑(d)/d) g,
  
  where g∈ℝ^4 is a Gaussian random vector such that g∼𝒩(0, I_4).
  
  
  
  
  
  
  
  
  
  
  
  
  
   §.§ Simulation studies
  
   In this section, we  perform extensive Monte Carlo simulations to study the finite-sample
  accuracy and power of our proposed statistics. For (<ref>), we not only report our results but also compare them with the existing statistics in the literature. We will call our statistics as .
  
  We start with (<ref>). In what follows regarding (<ref>), we use  to stand for the bootstrapping method using the Frobenius norm proposed in <cit.>,  to stand for the frequentist Bayes using the Frobenius norm proposed in <cit.>,   to stand for the sample splitting method using the Frobenius norm proposed in <cit.>,  to stand for the Le Cam optimal test proposed in <cit.>,  and  to stand for the bootstrapping method and the frequentist Bayes method respectively using the power-enhanced norm introduced in <cit.> with s_1=s_2=1 (see Definition 3.1 of <cit.>), and   and   the bootstrapping method  and the frequentist Bayes method respectively using the spectral norm. In the following discussion, we compare the performance of our  with all the aforementioned statistics. 
  
  
  In all the following simulations, we conduct 2,000 Monte-Carlo 
  repetitions for the bootstrapping and frequentist Bayes procedure. For the accuracy of the tests, we focus on reporting the results with the type I error rate 0.1 under different values of y=0.1, 1, 10 and various choices of the spikes. Moreover, we consider different scenarios to illustrate the usefulness and generality of our results.
  
  
    * Scenario I: We consider the case r_0=3 with d_1=d+7, d_2=7 and d_3=5, where d takes a variety of values. We consider the hypothesis testing for the eigenspace of Σ=I+ ∑_i=1^3d_i e_i e_i^* with ℐ={1,2}, where the null is 
  
      Z_0=e_1 e_1^*+e_2 e_2^*.
  
   In this scenario, the spiked eigenvalues are simple. We will consider the standard Gaussian distribution and the two-point distribution 1/3δ_√(2)+2/3δ_-1/√(2) as the distribution of entries of X. 
  
    * Scenario II:  We consider the case r_0=3 with d_1=d_2=d+5 and d_3=5, where d takes a variety of values. We consider the hypothesis testing under the null (<ref>).
  In this scenario, we have multiple/identical d_i's. We  consider both the standard Gaussian random variables and the two-point distribution 1/3δ_√(2)+2/3δ_-1/√(2) as the distribution of entries of X.     
  
  We mention that the asymptotic distribution of our statistic (<ref>) under the null hypothesis of (<ref>) has been established in Examples <ref> and <ref> for Scenarios I and II, respectively. 
  
  
  For both of the above two scenarios, we consider the alternative
  
      Z_a=v_1(φ) v_1(φ)^*+ v_2(φ) v_2(φ)^*,
  
  where for φ∈ [0,π/2]
  
      v_1(φ)=cosφe_1+sinφe_4,  v_2(φ)=cosφe_2+sinφe_5.
  
  Note that φ=0 corresponds to the null case of (<ref>). It is easy to see that κ_4=-1.5 for the two-point distribution and κ_4=0 for standard Gaussian random variable. 
  
  
  
  For Scenario I, from Tables <ref>–<ref>, we find that our proposed statistic (<ref>) is very accurate even for smaller values of N and d. Moreover, our statistic reaches accuracy regardless of the values of y. In contrast, for the other methods in the literature, we find that all of them lose their accuracy when y increases (i.e. M increases). Moreover, we find that most of the methods are conservative except for the Le Cam test.  Finally, we find that some of the methods, especially the frequentiest Bayes method with Frobenius norm, spectral norm or the power-enhanced norm in <cit.> are also reasonably accurate for larger values of d when y is small. Since our results allow us to deal with identical d_i's, we report the simulation results  in Tables <ref>–<ref> for Scenario II. We find that our statistic (<ref>) is also very accurate and outperforms the other methods especially when either the d_i's are small or y is large.  In Appendix <ref>, we report the simulation results for random variables with two-point distribution in Tables <ref>–<ref> for Scenario I and Tables <ref>–<ref> for Scenario II. We can get analogous conclusions.
  
  
  In summary, our proposed statistic (<ref>) is quite accurate for different values of d satisfying Assumption <ref>, even for smaller and multiple/identical ones. This accuracy is robust against different values of y.  As summarized in <cit.>, all the  previous methods in the literature request that M ≪ N. Therefore, when y increases (i.e., M diverges faster), we find that our methods perform better than all the other methods. Indeed, all our current results can be extended to the regime log M ≍log N following the discussion of <cit.>. We will pursue this direction in the future work. Moreover, since the computational complexity of the aforementioned methods depends on the a polynomial order of the dimensionality M (see <cit.>), they can be computationally intensive as M diverges faster. In contrast, our method works faster since it only depends the sample eigenvalues and eigenvectors.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  Then we compare the power of the above statistics under the alternative (<ref>) for different values of φ regarding Scenarios I and II and d=5, 50, respectively,  in Figures <ref>–<ref>.  
  We find that our method is very powerful even when y becomes larger and d  becomes smaller, and it outperforms the other methods especially when either y is large or d is small. When y is small and d is large, even though 
  for larger values of φ, many methods can obtain high power, we find that our proposed statistic (<ref>) is quite powerful even under a relatively weak alternative, i.e., smaller values of φ. We mention that the high power of  is not trustful since we have seen from the simulated type I error rates that it is not accurate. Similar results can be found for two-point random variables in Figures <ref>-<ref> of Appendix <ref>.  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  Next, we investigate the size and power of the orthogonal test (<ref>) using our proposed statistic (<ref>) with critical values generated adaptively according to Corollary <ref>. We conduct numerical simulations using the following two scenarios. 
  
  
  
    * Scenario A: We consider the case r_0=3 with d_1=d+7, d_2=7 and d_3=5, where d takes a variety of values. We consider the hypothesis testing of the eigenspace of Σ=I + ∑_i=1^3d_i e_i e_i^* with ℐ={1,2}, where the null is 
  
      Z_0=e_3 e_3^*+e_4 e_4^*.
  
  In this scenario, the spiked eigenvalues are simple. We will consider the standard Gaussian random variables and the two-point distribution 1/3δ_√(2)+2/3δ_-1/√(2).
  
    * Scenario B:  We consider the case r_0=3 with d_1=d_2=d+5 and d_3=5, where d takes a variety  of values. We consider the hypothesis testing under the null (<ref>).
  In this scenario, we have multiple/identical d_i. We  consider both the standard Gaussian random variables and the two-point distribution 1/3δ_√(2)+2/3δ_-1/√(2).     
  
  
  For both scenarios, we consider the alternative
  
      Z_a=v_1(φ) v_1(φ)^*+ v_2(φ) v_2(φ)^*,
  
  where for φ∈ [0,π/2]
  
      v_1(φ)=cosφe_3+sinφe_1,  v_2(φ)=cosφe_4+sinφe_2.
  
  Note φ=0 corresponds to the null case (<ref>). For Gaussian random variables, we report the simulated type-I error rates in Figure <ref> for Scenario A and in Figure <ref> for Scenario B for a variety of choices of y and d. The results for the two-point random variables can be found in Figures <ref> and <ref> of Appendix <ref>. We can see that our proposed statistic (<ref>)  with critical values generated adaptively according to Corollary <ref> are reasonably accurate even for smaller values of d and N and we  have better accuracy when N increases.  Finally, we examine the finite sample power for Gaussian  variables in Figures <ref> and <ref> and two-point variables in Figures <ref> and <ref>, we find that our statistic is powerful even for smaller values of φ (i.e., weaker alternative) and d. 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  § PRELIMINARIES
   
  In this section, we collect some basic notions and preliminary results which will be used in the proof of our main theorem. A key technical input is the isotropic local law from <cit.>. 
  
  
  
  
  
  
   §.§ Basic notions
  
  
  
  In the sequel,  we denote the Green function of Q by
  
      G(z):=(Q-z)^-1,    z∈ℂ^+.
  
  The matrix Q can be regarded as a finite-rank perturbation of the matrix H:=XX^*. In the sequel, we also need to consider ℋ:=X^*X which shares the same non-zero eigenvalues with H. We further denote the Green functions of H and ℋ respectively  by
  
      _1(z):=(XX^*-z)^-1   ,     _2(z):=(X^*X-z)^-1,    z∈ℂ^+,
  
  and their  normalized traces  by 
  
      m_1N(z):=1/MTr_1(z)=∫(x-z)^-1 dF_1N(x),
         m_2N(z):=1/NTr_2(z)= ∫(x-z)^-1 dF_2N(x),
  
  where  F_1N(x), F_2N(x) are the empirical spectral distributions of H and ℋ respectively, i.e.,
  
      F_1N(x):=1/M∑_i=1^M 1(λ_i(H)≤ x),    F_2N(x):=1/N∑_i=1^N 1(λ_i(ℋ)≤ x).
  
  Here we used λ_i(H) and λ_i(ℋ)  to denote the i-th largest eigenvalue of H and ℋ, respectively. 
  
  It is well-known since <cit.> that F_1N(x) and F_2N(x) converge weakly (a.s.) to the Marchenko-Pastur laws 
  ν_MP,1 and ν_MP,2 (respectively) given below
  
      ν_MP,1( dx):=1/2π xy√(((λ_+-x)(x-λ_-))_+) dx+(1-1/y)_+δ( dx),
         ν_MP,2( dx):=1/2π x√(((λ_+-x)(x-λ_-))_+) dx+(1-y)_+δ( dx),
  
  where λ_±:=(1±√(y))^2.  Note that here the parameter y may be N-dependent. Hence, the weak convergence (a.s.) shall be understood as ∫ g(x)  d F_aN(x)-∫ g(x) ν_MP,a( dx) a.s.⟶ 0 for any given bounded continuous function g:ℝ→ℝ, for a=1,2.  We further denote by F_α the cumulative distribution function of ν_MP,α for α=1,2. 
  Note that m_1N and m_2N can be regarded as the Stieltjes transforms of F_1N and F_2N, respectively.  We further define their deterministic counterparts, i.e.,  Stieltjes transforms of ν_MP,1,ν_MP,2,  by m_1(z),m_2(z),  respectively, i.e.,
   
      m_1(z):=∫ (x-z)^-1ν_MP,1( dx),    m_2(z):=∫ (x-z)^-1ν_MP,2( dx).
  
   From the definition (<ref>), it is elementary to compute 
   
      m_1(z)=1-y-z+√((λ_+-z)(z-λ_-))/2zy,  
         m_2(z)=y-1-z+√((λ_+-z)(z-λ_-))/2z,
  
  where the square root is taken with a branch cut on the negative real axis. Equivalently, we can also characterize m_1(z),m_2(z) as the unique solutions from ℂ^+ to ℂ^+ to the equations
  
      zym_1^2+[z-(1-y)]m_1+1=0,       zm_2^2+[z+(1-y)]m_2+1=0.
   
  Using (<ref>) and (<ref>), one can easily derive the following identities  
  
      m_1=-1/z(1+m_2),    1+zm_1=1+zm_2/y,   m_1((zm_2)'+1)=m_1'/m_1,
  
  which will be used in the later discussions. 
  
  
  
   
  
  
  
  
  
  
   §.§ Isotropic local law
  
  In this section, we state the isotropic local law from <cit.> together with some  consequences which will serve as the main technical inputs in the proofs. But before the statements of these estimates, we first introduce a basic lemma about  the notion “stochastic domination" introduced in Definition <ref>. 
    Let
  	
      X_i=(X_N,i(u):  N ∈ℕ,  u ∈U_N),  Y_i=(Y_N,i(u):  N ∈ℕ,  u ∈U_N),   i=1,2
  
  	be families of  random variables, where Y_i, i=1,2, are nonnegative, and U_N is a possibly N-dependent parameter set.	Let 
  	
      Φ=(Φ_N(u): N ∈ℕ,  u ∈U_N)
  
  	be a family of deterministic nonnegative quantities. We have the following results:
  	
  (i)	If X_1 ≺Y_1 and X_2 ≺Y_2 then X_1+X_2 ≺Y_1+Y_2 and  X_1X_2 ≺Y_1 Y_2.
  
  	
   (ii) Suppose X_1 ≺Φ, and there exists a constant C>0 such that  |X_N,1(u)| ≤ N^C a.s. and Φ_N(u)≥ N^-C uniformly in u for all sufficiently large N. Then 𝔼X_1 ≺Φ.
  
  
  
  	Part (i) is obvious from Definition <ref>. For any fixed ϱ>0, we have
  
      |𝔼 X_1|    ≤𝔼 |X_11(|X_1|≤  N^ϱΦ)|+𝔼 |X_11(|X_1|≥  N^ϱΦ)|
         ≤ N^ϱΦ+ N^C ℙ(|X_1|≥ N^ϱΦ)=O(N^ϱΦ)
  
  	for for sufficiently large N≥ N_0(ϱ). This proves part (ii). 
  
  
  In the sequel, we state the isotropic local law and related estimates.  We first introduce the following domain. For a small (but fixed) τ>0, we set
  
      𝒟≡𝒟(τ):={z=E+η∈ℂ:E≥λ_++N^-2/3+τ, 0<η≤τ^-1}.
  
  Conventionally,  for a=1,2,  we denote by 𝒢_a^l and 𝒢_a^(l)  the l-th power of 𝒢_a and the l-th derivative of 𝒢_a w.r.t. z, respectively. 
  With the above notation, we have the following theorem.
   
  Let  τ>0 in (<ref>) be a small but fixed constant. Let  , be complex deterministic unit vectors of proper dimensions. Suppose X satisfies Assumption <ref>. Then,  for any given l∈ℕ and α=1,2, we have  
  
      |⟨,_α^(l)(z)⟩-m_α^(l)(z)⟨,⟩|≺min{ (κ+η)^-2, (κ+η)^-1/4}/(κ+η)^l√(N),  
         |⟨,X^*_1^l+1(z)⟩|≺√( m_1(z)/Nη)1/(κ+η)^l,     |⟨,X_2^l+1(z) ⟩|≺√( m_2(z)/Nη)1/(κ+η)^l, 
         |m_α N^(l)(z)-m^(l)_α(z)|=O_≺(1/N(κ+η)^l+1)
      ,
  
  uniformly in z∈𝒟, where κ=E-λ_+. Further, when κ>K for some sufficiently large constant K>0 and |y-1|≥τ_0 for any small but fixed τ_0>0,  (<ref>) can be improved to 
  
      |m_α N^(l)(z)-m^(l)_α(z)|=O_≺(1/N(κ+η)^l+2).
  
  
  
  The case of l=0 of (<ref>) and  (<ref>) is  from the isotropic law for extended spectral domain in  Proposition 3.8 of <cit.> which is derived from Theorem 3.12 of <cit.> and the anisotropic laws in Theorem 3.7 of   <cit.>.  We emphasize that the extension to all z for (<ref>) is not directly included in Proposition 3.8 of <cit.>. Nevertheless, the approach used there can be adapted to prove  (<ref>) for all z, starting from the result in <cit.>  which is stated for fixed z only. Specifically, writing 2⟨,X^*_1(z)⟩=(X+)^*_1(z)(X+)-(X)^*_1(z)(X)-^*_1(z), one can use the approach for the extension in <cit.> to each symmetric quadratic forms and conclude the extension of the estimate for ⟨,X^*_1^l(z)⟩. 
  For other l≥ 1, we can derive the estimate easily from the case l=0 by using Cauchy integral with  the radius of the contour taking value |z-λ_+|/2 ≍κ+η . We also remark here that the original isotropic local laws in <cit.> were stated in much larger domains which also include the bulk and  edge regimes of the MP law. But here we only need the result for the domain far away from the support of the MP law. The (<ref>) can be obtained by the rigidity estimates of eigenvalues in <cit.>  and the definition of the Stieltjes transform;  see e.g. <cit.> for more details.  For (<ref>), denoting by F_α the cumulative distribution function of the measure ν_MP,α and using integration by parts, we have
  
      m_α N^(l)(z)-m^(l)_α(z)    ≍∫1/(λ-z)^ℓ+1 d (F_α N(λ)-F_α(λ))
         ≍∫1/(λ-z)^ℓ+2  (F_α N(λ)-F_α(λ))  dλ
         =O_≺(1/N∫_0^λ_++N^-2/3+τ/21/|λ-z|^ℓ+2 dλ ),
  
  where the last step follows from the rigidity results in Lemma <ref> below and its consequence on the convergence rate,  sup_x|F_α N(x)-F_α(x)|≺1/N; see for instance <cit.>. 
  Then  (<ref>) follows by elementary calculation. 
  
  
  
  
  
  Further,  in the following lemma, we collect some basic estimates of m_1 and m_2 which can be verified by elementary computations.
  
  
  
  
   
   Recall the definition of m_1 and m_2 in m1m2. For a=1,2, we have
  
      |zm_a(z)|≍ 1,    |1+zm_a(z)|≍min{1,(κ+η)^-1},
          m_a(z) ≍η/|z|^3/2√(κ+η),
  
  and 
  
      |z^3/2m_a^(l)(z)|≍ (κ+η)^-l+1/2,
         
      |(zm_a)^(l)| ≍(κ+η)^-l+1/2/|z|^3/2,   for  l≥ 1
  
  uniformly in z∈𝒟 for some positive constant C, where κ=E-λ_+.
  
  
  
   
  By Theorem <ref>, Lemma <ref> and Lemma <ref> , we can easily bound  (κ+η)^l z^*_1^(l), (κ+η)^l z^*X^*_1^(l) and (κ+η)^l^*X^* _1 ^(l)X for any deterministic unit vectors 𝐮 and 𝐯 of appropriate dimensions. For instance, for (κ+η)^l^*X^*_1 ^(l)X, we have the following estimates on 𝒟,
  
      ^*X^*_1 X =(1+zm_2) ^*+ O_≺(min{ (κ+η)^-2, (κ+η)^-1/4}N^-1/2)
                   = O_≺(min{1,(κ+η)^-1}),
         (κ+η)^l^*X^*_1 ^(l)X =(κ+η)^l^* (_1^l+z_1^l+1)
                   = O_≺(|(κ+η)^l (zm_2)^(l)^* |)=O_≺(|z|^-3/2√(κ+η) ).
  
  
  
  
  
  Using the isotropic local law, one can also get the following result, which gives the location of the outlier and the  extremal non-outlier.
  
  
  (Theorem 2.3 of <cit.>) Under Assumption <ref> and asd, we have for i∈ 1, r_0
  
      |μ_i-θ(d_i)|≺  (d_i-√(y))^1/2d_i^1/2 N^-1/2,     |μ_r_0+1-λ_+|≺ N^-2/3,
  
  where
  
      θ(z):=1+z+y+yz^-1,     for    z∈ℂ,   z>√(y).
  
  
  
  
  Further, for the eigenvalues of H,  we have the following rigidity estimate. 
    Suppose that Assumption <ref> holds. Denote by γ_i the i-th largest N-quantile of F_1, i.e, 1-F_1(γ_i)=i-1/2/N. We have  
  
      |λ_i(H)-γ_i|≺  N^-2/3(min{min{N,M}+1-i,i})^-1/3,
  
  for  1≤ i≤ (1-τ_0)min{N,M} or  for all 1≤ i≤min{N,M} in case |y-1|≥τ_0,  for some small but fixed τ_0>0. 
  Especially, (<ref>) implies
  
      |λ_1(H)-λ_+|≺ N^-2/3.
  
  Also, we note that λ_i(H)=0 if i≥min{N,M}+1.  
  
  
  
  
  
   
  By setting z:=θ(d_i)+ N^-K for a sufficiently large integer K, we have the estimates  
  
      |z| ≍θ(d_i) ≍ d_i,   κ+η≍κ≍Υ.
  
  Here we set 
  
      Υ≡Υ(d_i):=(d_i-√(y))^2/d_i .
   
  Applying the above estimates, we can further simplify the error bounds in Theorem <ref> to get 
  
      |⟨, Υ^l_α^(l)(z)⟩- Υ^lm_α^(l)(z)⟨,⟩|=O_≺(Δ(d_i)N^-1/2), 
         max{ |⟨, Υ^l-1X^*_1^l(z)⟩|, |⟨, Υ^l-1X_2^l(z)⟩| }
                     =O_≺( (d_i-√(y))^-1/2d_i^-1/2 N^-1/2), 
         |Υ^lm_α N^(l)(z)-Υ^lm^(l)_α(z)|=O_≺((d_i-√(y))^-2 N^-1)  ,
  
  for any deterministic unit vectors , of appropriate dimensions, where for convenience, we set 
  
      Δ(d):=min{(d-√(y))^-4d^2, (d-√(y))^- 1/2d^1/4}.
  
  We remark here that due to the Lipschitz continuity of _α^(l)(z), m_α^(l)(z), _α^l(z),  the estimates in (<ref>)-(<ref>) also hold for z=θ(d_i) with the error bounds unchanged.
  
  
  
  According to the definition of Δ(d), by simple calculations,  we see that for i∈ 1, r_0, i.e. d_i>√(y), 
  
      Δ(d_i)   ={[     (d_i-√(y))^- 1/2d_i^1/4 if  d_i<(1/2+√(√(y)+1/4))^2;          (d_i-√(y))^-4d_i^2 if  d_i≥(1/2+√(√(y)+1/4))^2 ]. 
         ≍{[             (d_i-√(y))^- 1/2  if  d_i<(1/2+√(√(y)+1/4))^2;                       d_i^-2 if  d_i≥(1/2+√(√(y)+1/4))^2. ].
  
  
  
  Finally, for the large d_i regime, i.e., d_i>K for some sufficiently large K, we will also need a convergence rate of the so-called eigenvector empirical spectral distribution (VESD), which is recently obtained in 
  <cit.>. For simplicity, we  state the result with necessary modification to adapt to our assumption. We also refer to <cit.> for the original statement under more general assumption. 
  
   Recall the notation λ_i(H) for the i-th largest eigenvalue of H=XX^*. We further denote by ϕ_i the unit eigenvector of H associated with λ_i(H).  
    Then, for a fixed unit vector , we define the so-called eigenvector empirical spectral distribution (VESD) with respect to  by 
   
      F_1N^ (x) = ∑_i=1^M | ϕ_i, |^2   1 (λ_i(H)≤ x).
   
  With the definition, we now introduce the following theorem  which gives the convergence rate of VESD. 
   
   Under the Assumption <ref> and the additional restriction |y-1|≥τ_0 with any small but fixed τ_0>0, for a given deterministic  vector ∈ S^M-1_ℝ, we have 
   
      sup_x | F_1N^ (x) - F_1(x)| ≺ N^-1/2,
  
   where F_1 is the cumulative distribution function of ν_MP,1 defined in 19071801.
  
  
  
  
  
  
  
  
  
  
   §.§ Auxiliary lemmas
   
  The following cumulant expansion formula plays a central role in our computation, whose proof can be found in <cit.> or <cit.>, for instance.
  
  
  (Cumulant expansion formula) For a fixed ℓ∈ℕ,  let f∈ C^ℓ+1(ℝ). Supposed ξ is a centered random variable with finite moments  to order ℓ+2.  Recall the notation κ_k(ξ) for  the k-th cumulant of ξ.  Then we have  
  
      (ξ f(ξ))=∑_k=1^ℓκ_k+1(ξ)/k!(f^(k)(ξ))+(r_ℓ(ξ f(ξ))),
  
  where the error term r_ℓ(ξ f(ξ)) satisfies
  
      |(r_ℓ(ξ f(ξ)))|≤   C_ℓ(|ξ|^ℓ+2)sup_|t|≤ s|f^ℓ+1(t)|
         +C_ℓ(|ξ|^ℓ+21(|ξ|>s))sup_t∈ℝ|f^ℓ+1(t)|
  
  for any  s>0 and C_ℓ satisfied C_ℓ≤ (Cℓ)^ℓ/ℓ! for some constant C>0.
  
  
  
  Next we collect some basic identities for the Green functions in (<ref>) without proof. 
  
  
  
   
  For any integer l≥1, we have
  
      _1^l=1/(l-1)!∂^l-1_1/∂ z^l-1 = 1/(l-1)!_1^(l-1),
         _1^lXX^*=_1^l-1+z_1^l,    X^*_1^lX=_2^lX^*X=_2^l-1+z_2^l.
  
  
  
  
   Further, for a ∈ 1, M and b∈ 1, N, we denote by  E_ab the M× N matrix with entires (E_ab)_cd=δ_acδ_bd. Let
  
      𝒫_0^ab=E_ab(E_ab)^*,  𝒫_1^ab=E_abX^*,  𝒫_2^ab=X(E_ab)^*.
  
  For any integer l≥1, it is also elementary to compute that 
  
      ∂_1^l/∂ x_ab=-∑_α=1^2∑_cl_1,l_2≥ 1
       l_1+l_2=l+1 _1^l_1𝒫_α^ab_1^l_2.
  
  Repeatedly applying the  identity derivative, we can  get the formulas for higher order derivatives of _1^l w.r.t.  x_ab. Moreover, by (<ref>) and  the product rule, we can easily deduce the derivatives of X^*_1^l w.r.t. x_ab.   For the convenience of the reader, we collect more basic formulas of the  derivatives of Green functions in Appendix <ref>.
  
  
  
  
  
  
  § GREEN FUNCTION REPRESENTATION
  
  In this section, we  express  , P_𝖨 in terms of  the Green function _1(z) in (<ref>).  And for singleton 𝖨={i}, we also express the eigenvalue μ_i in terms of  the Green function _1(z) in (<ref>). Both  representations are obtained via doing expansions of certain functionals of the Green function. The expansion for the eigenvalue in the multiple case can be done similarly, and the details will be stated in Appendix <ref>.  This representation will allow us to work with the Green function instead of the eigenvalue and eigenvector statistics.  We also remark here that similar derivation of the Green function representation has appeared in previous work such as <cit.>. But here for eigenvectors, we need to do it up to a higher order precision, in order to capture all contributing terms for the fluctuation.  For instance, when ∈Span{𝐯_j }_j∈ 1,M∖𝖨, the fluctuation of , P_𝖨 is one order smaller than that of the case ∈Span{𝐯_t }_t ∈𝖨. In order to cover the situation like the former case, we will need to investigate a higher order term in the expansion. 
  
  We start with a few more notations. The vector  is decomposed as
  
      w=∑_j=1^r ⟨w, _j⟩_j+,   where ∈Span{_1⋯,_r}^⊥.
  
  Hereafter,  we take (<ref>) as the definition of 𝐮. 
  
  We define the centered Green function by
  
      Ξ(z) := _1(z) - m_1(z) I,
   
  and introduce its quadratic forms
  
      χ_ij (z) = _i^* Ξ(z) _j,    χ_ j(z) = ^* Ξ(z) _j,     i,j∈ 1, r.
  
  For brevity, we further set
   
      w:= Σ^-1/2w=∑_j=1^r  w_j_j+  with   w_j:= ⟨w,_j ⟩/√(1+d_j).
  
  Also, for d>0, we define the following functions
  
      f(d):=1/d(d+1)(d^2-y),      g(d):=1/d (d+1)(d+y)(d^2-y) ,
  
  which are of order O(d(d-√(y))) and O(d^2(d-√(y)))  respectively.
  And for i∈ 1,r, we set for d≠ d_i, 
  
      ν_i (d) := d_i (d+1)/d_i -d,
  
  and further introduce the following shorthand notations
  
      δ_ij:=|d_i-d_j|,    δ_i0:=|d_i-√(y)|
  
  for i, j ∈ 1,r.
  
   
  With the above notations and _𝖨, ς_𝖨 from  (<ref>) and (<ref>), we have the following lemma. 
  Suppose that the assumptions of Theorem <ref> hold. 
  For i ∈ 1, r_0, we have 
  
      , P_𝖨=    d_i^2-y/d_i(d_i+y)‖_𝖨‖^2-2d_i(d_i+1)_𝖨^*Ξ(θ(d_i))_𝖨-2f(d_i)/√(1+d_i)ς_𝖨^*Ξ(θ(d_i))_𝖨
          -f(d_i)^2/1+d_i_𝖨^*Ξ'(θ(d_i))_𝖨+ g(d_i)∑_t ∈𝖨(_t^*Ξ(θ(d_i)) ς_𝖨)^2 
         -d_i(1+d_i)g(d_i)∑_j ∈𝖨^cd_j/(d_i-d_j)^2(_j^* Ξ(θ(d_i))_𝖨)^2
         +O_≺( N^-1/2-ε(δ_i0√(d_i) ‖ς_𝖨‖ + d_i ‖_𝖨‖) ‖_𝖨‖ Δ(d_i) ) 
         +O_≺(  N^-1-ε( d_i‖ς_𝖨‖^2 + d_i^2 ‖w_𝖨‖^2( ∑_j∈𝖨^cd_j/δ_ij) (  ∑_j∈𝖨^cd_i/δ_ij) ) δ_i0  d_i Δ(d_i)^2 )
  
  for some small fixed constant ε>0. 
  
  
  
   It will be seen that the bounds in weak_est_XG actually give the  true typical size of 
  the quadratic forms. In light of this, 
   Lemma <ref> suggests that  the  distribution of , P_i is ultimately governed by the joint distribution of the quadratic forms _𝖨^*Ξ_𝖨, ς_𝖨^*Ξ_𝖨, _𝖨^*Ξ' _𝖨, {_t^*Ξς_𝖨}_t∈𝖨, {_j^* Ξ_𝖨}_j∈𝖨^c, since the last two error terms in (<ref>) are smaller than the sum of the second to the fifth fluctuating terms in any case, with high porbability.  Here we drop the parameter θ(d_i) from Ξ for simplicity.
  
  
   
  
   
   Recall (<ref>) together with (<ref>).  We rewrite   S as
  
      S=V(d_1,⋯,d_r)V^*,
   
  by setting V=(_1,⋯,_r). Therefore, we have 
   
      Σ^-1=I-VDV^*.
  
   with
   
      D=(d_1/1+d_1,⋯,d_r/1+d_r).
  
    From Lemma <ref>, we observe that all μ_i, i∈𝖨 tend to the identical limit θ(d_i) with error bound O_≺ (N^-1/2(d_i-√(y))^1/2).
   
    Let Γ_i be the boundary of a disc centered at d_i with  radius ρ_i,
    
      ρ_i:=1/2( min_j∈𝖨^c|d_i-d_j| ∧ (d_i-√(y))),
  
      such that the disc is away from the critical value √(y) and other distinct d_j's.
    Therefore under the definitions of ρ_i and θ(z) in def. of theta,  with  Assumption <ref>, the contour θ(Γ_i), which is the image of Γ_i under the map θ(·), encloses exactly |𝖨| eigenvalues of Q, i.e. μ_i,i∈𝖨. This follows from the fact 
    
      |θ(d_i+ρ_i)-θ(d_i)| ≍ (d_i-√(y))ρ_i > (d_i-√(y))^1/2 N^-1/2+ ε,  for some ε >0,
  
  where the last inequality is guaranteed  by   Assumption <ref>. 
   According to Lemma  <ref>, together with the Cauchy integral, we have the following equality with high probability 
  
      , P_𝖨=-1/2π∮_θ(Γ_i)w^*G(z)w  dz.
   
   With the notations for V, S, D and Σ^-1, using the setting (<ref>), we can write 
  
      G(z)   =( Σ^1/2 XX^*  Σ^1/2 - z I )^-1= Σ^-1/2( _1^-1(z) +z Σ^-1/2 S Σ^-1/2)^-1Σ^-1/2
          = Σ^-1/2( _1^-1(z) + z VDV^* )^-1Σ^-1/2.
  
  Then, it follows from the matrix inversion lemma that
  
      G(z)   = Σ^-1/2_1(z) Σ^-1/2-z Σ^-1/2_1(z) V(D^-1+zV^*_1(z) V)^-1 V^*_1(z) Σ^-1/2.
  
  With the notation introduced in (<ref>), we can further write
   
      w^*G(z)w=w^*_1(z)w-zw^*_1(z)V(D^-1+zV^*_1(z)V)^-1V^*_1(z) w.
  
  Plugging (<ref>) into (<ref>), and noticing that the contour integral of w^*_1(z)w on θ(Γ_i) is zero with high probability by Assumption <ref> and the rigidity of eigenvalues of H ((<ref>)), one has
    
      , P_𝖨 =1/2π∮_θ(Γ_i) zw^*_1(z)V(D^-1+zV^*_1(z)V)^-1V^*_1(z) w  dz
  
  with high probability. 
  
  For the integrand in (<ref>), we first recall the notation in (<ref>) and then 
  we apply resolvent expansion 
  
      (D^-1+zV^*_1(z)V)^-1 
      =   L(z) - z L(z) V^*Ξ(z)V L(z) 
         + ( z L(z)V^*Ξ(z)V )^2 (D^-1+zV^*_1(z)V)^-1,
  
  where 
  
      L(z) := ( D^-1 + z m_1(z)  )^-1.
  
  With (<ref>), we can further rewrite  (<ref>)  as
  
      , P_𝖨=1/2π∮_θ(Γ_i) z ( m_1(z) ^*V +^* Ξ(z) V )( L(z) - z L(z) V^*Ξ(z)V L(z)
             + ( z L(z)V^*Ξ(z)V )^2 (D^-1+zV^*_1(z)V)^-1) ( m_1(z) V^*  + V^* Ξ(z) )   dz.
  
  Hence,  we  can  write
  
      , P_𝖨 = S_1 + S_2 + S_3 +R,
  
  by defining
   
      S_1:= 1/2π∮_θ(Γ_i) z m_1^2(z) ^* V L(z) V^*   dz, 
         S_2:= 1/2π∮_θ(Γ_i)( 2 z m_1(z)^* V L(z) V^* Ξ(z)  - z^2 m_1^2(z) ^* V L(z) V^* Ξ(z) V L(z) V^* )   dz, 
         S_3:=  1/2π∮_θ(Γ_i)( z  ^* Ξ(z) V L(z) V^* Ξ(z)  - 2 z^2 m_1(z) ^* (VL(z)V^* Ξ(z))^2 
                         + z^3 m_1^2(z) ^* (VL(z)V^* Ξ(z))^2V L(z) V^* )   dz,
  
  and we take eq:decomposew as the definition of the remainder term R. It remains to estimate S_1, S_2, S_3 and R.
  From the definitions in (<ref>) and (<ref>),  
  it is easy to check the identity
  
      1+z^-1 + θ(z) m_1 (θ(z))=0.
   
  With the above identity, we see that
  
      V L(θ(z)) V^* = ∑_j=1^r  _j _j^*/1+d_j^-1 + θ(z) m_1 (θ(z)) = ∑_j=1^r   z d_j _j _j^*/z- d_j.
  
  Therefore, by the residue theorem,
   
      S_1    = 1/2π∮_Γ_iθ(z) θ'(z) m_1^2(θ(z)) ^* V L(θ(z)) V^*   dz 
         = θ(d_i) θ'(d_i) m_1^2 (θ(d_i)) d_i^2 ∑_t∈𝖨(^* _t)^2 = d_i^2 - y /d_i (d_i+ y)∑_t∈𝖨(^* _t)^2.
  
  Similarly, using 19071910, we can get
  
      S_2    = 1/2π∮_Γ_i( 2θ(z) θ'(z) m_1(θ(z))  ∑_j=1^r z d_j/z- d_j (^* _j) ^* Ξ(θ(z)) _j 
           - θ^2(z) θ'(z) m_1^2(θ(z)) ∑_j,k=1^r z^2 d_j d_k/(z-d_j)(z-d_k) (^* _j) (^* _k) _j^*Ξ(θ(z)) _k )   dz.
   
  Further by the residue theorem together with the definition of θ and m_1 in (<ref>) and (<ref>), we can get
  
      S_2   =-2d_i (d_i+1)^2 ∑_j,k∈𝖨 w_j w_kχ_jk(θ(d_i)) - 2 f(d_i) ∑_j∈𝖨^cν_i(d_j)  w_j ∑_t∈𝖨 w_t χ_tj(θ(d_i)) 
           - 2 f(d_i) ∑_t∈𝖨 w_t χ_ t(θ(d_i)) - f(d_i)^2 ∑_j,k∈𝖨 w_j w_kχ_jk'(θ(d_i))
         =-2d_i(d_i+1) _𝖨^*Ξ(θ(d_i))_𝖨-2 f(d_i)/√(1+d_i)_𝖨^*Ξ(θ(d_i))ς_𝖨 - f(d_i)^2/1+d_i_𝖨^*Ξ'(θ(d_i))_𝖨,
  
   where we recall the notations in 19071920, 19071921 and def:v_I_varsigma_I. 
   Next, we fix a sufficiently large K>0. For the case d_i≤ K, by isotropic local law  weak_est_DG, we can bound both the first and third term on the RHS of weak_19071941  by O_≺ ( ∑_t∈𝖨 | w_t|^2  Δ(d_i) N^-1/2). In the case d_i>K,  this  crude bound shall be refined to O_≺ ( ∑_t∈𝖨 | w_t|^2  d_i^3Δ(d_i) N^-1/2). Here in the latter case we carry the additional factor d_i^3 since it can be very large; for instance, when d_i∼ N^C. Nevertheless,  a more precise bound can be obtained for the combination of the first and third terms (<ref>) due to a hidden cancellation in case d_i>K.  In order to see this cancellation, we need to apply in Theorem <ref>. To this end,  we first set the following notation for the normalized vector  
  
      _𝖨^0:=            
      _𝖨/_𝖨,    if _𝖨≠0;
      0,    otherwise.
  
  Note that due to the triviality of the case _𝖨=0, we only discuss the case _𝖨≠0 in the sequel. 
  
  Recall the definition of  the VESD from def:VESD0 with 𝐯=_𝖨^0 
  
      F_1N^w_𝖨^0 (x) = ∑_i=1^M | ϕ_i, _𝖨^0|^2  11 (λ_i(H)≤ x),
   
  and we denote by F_1(x) the distribution function of ν_MP,1( dx) in 19071801.
  By Theorem <ref>, we obtain,
  
      sup_x| F_1N^w_𝖨^0(x) - F_1(x)  | = O_≺ (N^-1/2).
  
  In light of the fact that _𝖨^0 is the normalized _𝖨, we can write the combination of the first and third terms on the RHS of (<ref>) as
  
      |-2d_i(d_i+1) _𝖨^*Ξ(θ(d_i))_𝖨 - f(d_i)^2/1+d_i_𝖨^*Ξ'(θ(d_i))_𝖨| 
          = ‖_𝖨‖^2 |∫(-2d_i(d_i+1)/x-θ(d_i)  - (1+d_i)(d_i^2-y)^2/d_i^2/(x-θ(d_i))^2) d(F_1N^w_𝖨^0 (x)- F_1 (x))| 
         ≍‖_𝖨‖^2 |∫^λ_+ + N^-2/3 + τ/2_λ_- - N^-2/3 + τ/2(2d_i(d_i+1)/(x-θ(d_i))^2  + 2(1+d_i)(d_i^2-y)^2/d_i^2/(x-θ(d_i))^3)   (F_1N^w_𝖨^0 (x)- F_1 (x)) dx |,
  
  where in the last step, we used integration by parts, together with the rigidity results in Lemma <ref>. 
  In the case d_i>K for sufficiently large K, by the definition of θ(d_i), we see that θ(d_i) ∼ d_i and thus θ(d_i) is also sufficiently large. Hence,  applying Taylor expansion for both 1/(x-θ(d_i))^2, 1/(x-θ(d_i))^3 at x=0, we can easily get the bound 
  
      2d_i(d_i+1)/(x-θ(d_i))^2  + 2(1+d_i)(d_i^2-y)^2/d_i^2/(x-θ(d_i))^3 = O(1/d_i)
   
  uniformly for x∈ [λ_- - N^-2/3 + τ/2, λ_+ + N^-2/3 + τ/2]. Then using cancel:Taylor and bdd:Koldis_VESD, we finally get the estimate 
  
      |-2d_i(d_i+1) _𝖨^*Ξ(θ(d_i))_𝖨 - f(d_i)^2/1+d_i_𝖨^*Ξ'(θ(d_i))_𝖨| =  O_≺(∑_t∈𝖨 | w_t|^2 N^-1/2).
  
  We therefore get a unified estimate O_≺(∑_t∈𝖨 | w_t|^2 d_i^2 Δ(d_i) N^-1/2) for this combination, no matter d_i≤ K or d_i>K.
  
  
  
  Further, from the isotropic local law weak_est_DG,  Assumption <ref>, and the definition of _𝖨, ς_𝖨 in def:v_I_varsigma_I, we get the following crude bound  
  
      S_2   =O_≺( ∑_t∈𝖨| w_t| Δ(d_i) N^-1/2( d_i^2 ∑_t∈𝖨| w_t| + ∑_j∈𝖨^c | w_j| d_i^2d_jδ_i0/δ_ij  + d_iδ_i0‖‖))
         =O_≺ (Λ_2),
  
  where we recall the notations δ_ij, δ_i0 in def:d_i-d_j and  define 
  
      Λ_2   :=( ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_j∈𝖨^c| w_j|d_id_j/δ_ij +  ‖‖)  (∑_t∈𝖨| w_t|d_i/δ_i0)  δ_i0^2 Δ(d_i) N^-1/2.
  
  Since Λ_2 can degenerate depending on the size of ∑_t∈𝖨| w_t|, it is necessary to also consider the fluctuation of the higher order term S_3. We shall obtain more precise estimates for the summands in S_2 in later context, after first obtaining an estimate of S_3.
  
  
  Next, we turn to estimate S_3 (c.f. (<ref>)). We estimate the integrals of three terms in the integrand separately. First, using the residue theorem together with the notations in (<ref>) and (<ref>), we have  
   
      1/2π∮_θ(Γ_i)  z  ^* Ξ(z) V L(z) V^* Ξ(z)   dz=θ(d_i)θ'(d_i)d_i^2 ∑_t∈𝖨(^*Ξ(θ(d_i))_t)^2 
           = g(d_i)∑_t∈𝖨( ∑_j,k∈𝖨^c w_j  w_k χ_tj(θ(d_i)) χ_tk(θ(d_i)) +2 ∑_j∈𝖨^c w_j χ_tj(θ(d_i)) χ_ t(θ(d_i)) + χ_ t^2(θ(d_i)))
            +O_≺(d_i^2(d_i-√(y))Δ(d_i)^2∑_t∈𝖨| w_t|‖‖N^-1),
  
  where the error bound follows from the isotropic local law weak_est_DG and the  expression for g(d).
  
  For the second part of the integral, we have
   
      1/2π∮_θ(Γ_i) - 2 z^2 m_1(z) ^* (VL(z)V^* Ξ(z))^2   dz 
         =-2 θ^2(d_i)θ'(d_i)m_1(θ(d_i)) ∑_j∈𝖨^c,   t∈𝖨d_i^3 d_j/d_i - d_j (^*_j) ^*Ξ(θ(d_i))_tχ_tj(θ(d_i)) 
            -2 θ^2(d_i)θ'(d_i)m_1(θ(d_i)) ∑_j∈𝖨^c,  t∈𝖨d_i^3 d_j/d_i - d_j (^*_t)^*Ξ(θ(d_i))_jχ_tj(θ(d_i)) 
           -2d_i^2 ∑_t, k∈𝖨(^*_t) (θ^2(z)θ'(z)m_1(θ(z))z^2 ^*Ξ(θ(z))_k χ_tk(θ(z)) )'|_z=d_i
         =2g(d_i)∑_j∈𝖨^c(1+d_i)d_j/d_i-d_j w_j ∑_t∈𝖨χ_tj(θ(d_i))(∑_k∈𝖨^c w_k χ_tk(θ(d_i)) +χ_ t(θ(d_i)) ) 
           +O_≺(∑_t∈𝖨, j∈𝖨^c(d_i^3d_jδ_i0/δ_ij + d_i^4)| w_t|‖‖Δ(d_i)^2 N^-1).
  
  Similarly, we used isotropic local law weak_est_DG for l=0,1 and the definition for g(d) in (<ref>)  for the estimates in the last step.
  
  
  
  
  Analogously, for  the last term, we obtain
   
      1/2π∮_θ(Γ_i) z^3 m_1^2(z) ^* (VL(z)V^* Ξ(z))^2V L(z) V^*   dz 
         =g(d_i)∑_j,k∈𝖨^c(1+d_i)^2d_jd_k/(d_i-d_j)(d_i-d_k) w_j  w_k ∑_t∈𝖨χ_tj(θ(d_i)) χ_tk(θ(d_i))
          +2g(d_i)∑_j,k∈𝖨^c(1+d_i)^2d_jd_k/(d_i-d_j)(d_i-d_k) w_kχ_jk(θ(d_i)) ∑_t∈𝖨 w_tχ_tj(θ(d_i)) 
         +2∑_t,k∈𝖨, j∈𝖨^c w_t  w_j d_i^2d_j(θ(z)^3θ'(z)m_1(θ(z))^2z^3/z-d_jχ_tk(θ(z))χ_kj(θ(z)) )'|_z=d_i
         +∑_t,k∈𝖨, j∈𝖨^c w_t  w_k d_i^2d_j (θ(z)^3θ'(z)m_1(θ(z))^2z^3/z-d_jχ_tj(θ(z))χ_kj(θ(z)) )'|_z=d_i
         +∑_t, j, k∈𝖨 w_t  w_kd_i^3 (θ(z)^3θ'(z)m_1(θ(z))^2z^3χ_tj(θ(z))χ_kj(θ(z)) )”|_z=d_i.
  
  In the sequel, we will keep the first term on the RHS of (<ref>) as it is and estimate the other terms. 
  First, notice that the second term  on the RHS of (<ref>) can be estimated by 
  
      2g(d_i)∑_j,k∈𝖨^c(1+d_i)^2d_jd_k/(d_i-d_j)(d_i-d_k) w_kχ_jk(θ(d_i)) ∑_t∈𝖨 w_tχ_tj(θ(d_i)) 
         =O_≺(∑_j,k∈𝖨^c, t∈𝖨  | w_t| | w_k| d_i^4d_jd_kδ_i0/δ_ijδ_ikΔ(d_i)^2N^-1),
  
  Similarly, simple calculation shows that the third term on the RHS of (<ref>) can be bounded crudely by   
  
      ∑_t,k∈𝖨, j∈𝖨^c w_t w_j d_i^2d_j[(θ”(d_i)d_i^4 +θ'(d_i) ^2 d_i^3+ θ'(d_i) d_i^3/d_i-d_j + θ'(d_i) d_i^4  /(d_i-d_j)^2 )χ_tk(θ(d_i))χ_kj(θ(d_i)) 
         + θ'(d_i) ^2 d_i^4/d_i-d_j(χ_tk'(θ(d_i))χ_kj(θ(d_i)) + χ_tk(θ(d_i))χ_kj'(θ(d_i)))] 
         =O_≺(∑_t∈𝖨, j ∈𝖨^c| w_t| | w_j| (d_i^5d_j/δ_ij + d_i^5d_jδ_i0/δ_ij^2) Δ(d_i)^2N^-1).
  
  
  Next,   we turn to  the last term on the RHS of (<ref>). We split the discussion into two cases: d_i≤ K or d_i>K, for some sufficiently large (but fixed) K>0. For d_i≤ K , it is easy to compute the derivative by chain rule and use isotropic local law weak_est_DG to get the bound O_≺( ∑_t,k∈𝖨 | w_t| | w_k| δ_i0^-1Δ(d_i)^2N^-1) directly for the last term on the RHS of (<ref>), and thus we omit the detail. In case d_i>K,  we first introduce a new form of eigenvector empirical spectral distribution(VESD) of H with respect to two fixed unit vectors ,, which is defined by 
  
      F_1N^,(x)  = ∑_i=1^M  ϕ_i, ϕ_i, 11 (λ_i(H)≤ x).
  
  Correspondingly, we set F_1^, (x) = ,    F_1(x). Then by triangle inequality and Theorem <ref>, we have  the estimate
  
      sup_x| F_1N^,(x)- F_1^, (x) |    ≤1/2‖+‖^2 sup_x| F_1N^(+)^0(x) - F_1(x)| 
         +1/2sup_x |F_1N^(x) - F_1(x)| + 1/2sup_x|F_1N^ - F_1|
          = O_≺(N^-1/2).
  
  Here we use (+)^0 to denote the normalized +. 
  Similarly to def:VESD- 2020072106, after computing the derivative of  the last term on the RHS of (<ref>), we will see the hidden cancellations 
   among the resulting terms using the following estimates,
   
      2χ_tj(θ(z)) + θ(d_i)χ_tj'(θ(z))  = O_≺(d_i^-3 N^-1/2),  
         χ_tj(θ(z)) χ_kj'(θ(z)) θ”(d_i) = O_≺(d_i^-4Δ(d_i)^2N^-1/2), 
         6χ_tj(θ(z)) + 6θ(d_i)χ_tj'(θ(z))  + θ(d_i)^2χ_tj”(θ(z)) = O_≺(d_i^-3 N^-1/2)
  
   which can be checked readily  by using (<ref>).
    Then after elementary computations,  
  we can get for the case d_i>K, the  last term on the RHS of (<ref>) admits the bound O_≺( ∑_t,k∈𝖨 | w_t| | w_k| N^-1). Combining the estimates for the two cases, d_i≤ K or d_i>K, we can uniformly bound  the last term on the RHS of (<ref>) by 
  
      O_≺( ∑_t,k∈𝖨 | w_t| | w_k| d_i^5δ_i0^-1Δ(d_i)^2 N^-1).
  
  
  
  Next, we turn to estimate the fourth term on the RHS of weak_d_is33, 
  which by product rule can be written up as 
  
      ∑_t,k∈𝖨, j∈𝖨^c w_t  w_k d_i^2d_j (θ(z)^3θ'(z)m_1(θ(z))^2z^3/z-d_jχ_tj(θ(z))χ_kj(θ(z)) )'|_z=d_i
         =-d_i(1+d_i)^2g(d_i)∑_t, k∈𝖨, j∈𝖨^c w_t  w_k d_j/(d_i-d_j)^2χ_tj(θ(d_i))χ_kj(θ(d_i))  
            +  ∑_t,k∈𝖨, j∈𝖨^c w_t  w_k d_i^2d_j /d_i-d_j(θ(z)^3θ'(z)m_1(θ(z))^2z^3χ_tj(θ(z))χ_kj(θ(z)) )'|_z=d_i.
  
  The first term on the RHS of 2020072201 can be further simplified to 
  
      -d_i(1+d_i)g(d_i)∑_ j∈𝖨^cd_j/(d_i-d_j)^2(_𝖨^*Ξ(θ(d_i))_j)^2.
  
  The above may exceed the stochastic bound Λ_2 defined in (<ref>). While a posteriori, it will be clear that Λ_2 gives the typical size of S_2. Hence,  we shall keep the  term (<ref>) explicit instead of grouping it into the errors. As for the second term of 2020072201,
  analogously to the last term on the RHS of weak_d_is33, by splitting the discussion into two cases based on the  size of d_i, we can  get the estimate 
  
      O_≺(∑_t,k∈𝖨, j ∈𝖨^c| w_t| | w_k| d_i^4d_j/δ_ijΔ(d_i)^2N^-1).
   
  
  
  
  Combining (<ref>)-(<ref>), after necessary simplification, we arrive at 
  
      S_3   =g(d_i)∑_t∈𝖨(_t^*Ξ(θ(d_i))ς_𝖨)^2- d_i(1+d_i)g(d_i)∑_ j∈𝖨^cd_j/(d_i-d_j)^2(_j^*Ξ(θ(d_i))_𝖨)^2
           +O_≺ (ℛ_3)
         =: S_3+ O_≺ (ℛ_3),
  
  where  
   
      ℛ_3   := (∑_t∈𝖨| w_t|d_i/δ_i0) ‖‖( ∑_j∈𝖨^cd_j/δ_ij+d_i/δ_i0) d_i^2δ_i0^2 Δ(d_i)^2N^-1
         +(∑_j∈𝖨^c| w_j|d_id_j/δ_ij) ( ∑_t∈𝖨| w_t|d_i/δ_i0)( ∑_k∈𝖨^cd_k/δ_ik + d_i/δ_i0 + ∑_k∈𝖨^cd_i/δ_ik)  d_i^2δ_i0 ^2Δ(d_i)^2N^-1
         + (∑_t∈𝖨| w_t|d_i/δ_i0)^2 (d_i/δ_i0+ ∑_j∈𝖨^cd_j/δ_ij)  d_i^2δ_i0^2Δ(d_i)^2N^-1.
  
  Now we claim that ℛ_3≺ N^-εΛ_2 for some small (but fixed) ε>0, where Λ_2 is defined in def:Lambda_2. This can be achieved by applying assumption  asd and asd2 and  using the following three estimates 
  
      d_i^3/δ_i0Δ(d_i) N^-1/2≺ N^-ε,    ∑_k∈𝖨^cd_i^3/δ_ikΔ(d_i) N^-1/2≺ N^-ε ,
         ∑_k∈𝖨^cd_i^2 d_k/δ_ikΔ(d_i) N^-1/2≺ N^-ε.
  
  The first two estimates in 2020011405 can be easily checked by combining the assumption for δ_i0=d_i-√(y) in asd, the non-overlapping condition  asd2  with  the definition of Δ(d_i) in def:Delta(d) and its estimates in  Remark <ref>. The last estimate in (<ref>) can be derived as follows
  
      ∑_kd_i^2 d_k/δ_ikΔ(d_i) N^-1/2≤∑_k( d_i^3/δ_ik+ d_i^2)Δ(d_i) N^-1/2
         ≤∑_k( d_i^3/δ_ik+ d_i^3/δ_i0)Δ(d_i) N^-1/2≺ N^-ε.
  
  Here for the first inequality, we use the triangle inequality d_k≤ d_i +δ_ik. The last inequality above holds from the definition of Δ(d_i) and the non-overlapping condition asd2 for δ_i=min_j∈𝖨^cδ_ij as well as the first estimate in 2020011405.
  
  Thus to see ℛ_3≺ N^-εΛ_2, we first observe that the second and third terms on the RHS of def:R_3 are much smaller than Λ_2 by applying the estimates in 2020011405 and the trivial fact d_i>δ_i0. For the first term on the RHS of def:R_3, notice that 
  
      ‖‖≤( ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_j∈𝖨^c| w_j|d_id_j/δ_ij +  ‖‖),
  
  which is a consequence of the fact  
  
      d_id_j/|d_i-d_j|>C
  
  for some constant C>0. Using 20200417 and the first and third estimates in  2020011405, we get the first term on the RHS of def:R_3 is also negligible compared to Λ_2. 
  To conclude, we have ℛ_3≺ N^-εΛ_2.
   
  
   Therefore, we can rewrite 
  
      S_3= S_3+O_≺(Λ_2/N^ε),
  
  for some small fixed positive ε. 
  Further, using isotropic local law weak_est_DG together with the definitions of w_𝖨, ς_𝖨 in   def:v_I_varsigma_I,
   we can crudely estimate the size  of S_3 in 2020011402 by O_≺ (Λ_3) where 
  
      Λ_3:=   ( ∑_j∈𝖨^c| w_j|d_j/δ_ij+‖‖/d_i)^2 d_i^4  δ_i0 Δ(d_i)^2N^-1
          +  ( ∑_t∈𝖨, j∈𝖨^c| w_t |d_j/δ_ij) ( ∑_t∈𝖨, j∈𝖨^c| w_t |d_i/δ_ij) d_i^4 δ_i0 Δ(d_i)^2N^-1.
  
  
  Now it remains to estimate the remainder term R in (<ref>). We shall prove that 
  
      |R|= O_≺( Λ_2+Λ_3/N^ε).
  
  Following the derivation of (<ref>), one has 
    
      R   =,  P_𝖨-S_1-S_2-S_3 
         =-1/2π∮_θ(Γ_i)zm_1(z)w^*V ( zL(z)V^* Ξ(z)V )^3 (D^-1+zV^*_1(z)V)^-1m_1(z) V^*w  dz
            +2/2π∮_θ(Γ_i)zm_1(z)w^*V( zL(z)V^* Ξ(z)V )^2 (D^-1+zV^*_1(z)V)^-1  V^*Ξ(z) w  dz
            -1/2π∮_θ(Γ_i)zw^* Ξ(z)V  ( zL(z)V^* Ξ(z)V )(D^-1+zV^*_1(z)V)^-1  V^*Ξ(z) w  dz
         =:T_1+T_2+T_3.
   
  The estimations of T_1, T_2, T_3 cannot be computed by only applying the residue theorem. Instead, we shall follow the method used in Lemma 5.6 of <cit.>. More specifically,
   for T_1, we first use the resolvent expansion formula 
  
      (D^-1+zV^*_1(z)V)^-1=L(z)-(D^-1+zV^*_1(z)V)^-1zV^*Ξ(z)V L(z)
   
  to split it into two parts
  
      T_1   =-1/2π∮_θ(Γ_i)zm_1(z)w^*V ( zL(z)V^* Ξ(z)V )^3 L(z)m_1(z) V^*w  dz
            +1/2π∮_θ(Γ_i)zm_1(z)w^*V ( zL(z)V^* Ξ(z)V )^3 
      (D^-1+zV^*_1(z)V)^-1V^*Ξ(z)V  L(z) m_1(z) V^*w  dz
         =:T_11+T_12.
  
  For T_11, we estimate it using the residue theorem. Similar to the previous calculations as in (<ref>)-(<ref>), one can derive that the magnitude of T_11 is stochastically bounded by 
  
      (∑_j∈𝖨^c| w_j| d_j/δ_ij) (∑_j∈𝖨^c| w_j| d_j/δ_ij + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij) ( ∑_j∈𝖨^c d_j/δ_ij)d_i^6δ_i0Δ(d_i)^3N^-3/2
         +(∑_j∈𝖨^c| w_j| d_j/δ_ij + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij)  (∑_j∈𝖨^c| w_j| d_j/δ_ij + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij+∑_t∈𝖨| w_t| d_i/δ_i0) d_i^6 Δ(d_i)^3N^-3/2
         + (∑_j∈𝖨^c| w_j| d_j/δ_ij^2 + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij^2) (∑_j∈𝖨^c| w_j| d_j/δ_ij + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij + ∑_t∈𝖨| w_t|/δ_i0) d_i^7 δ_i0Δ(d_i)^3N^-3/2
         +(∑_j∈𝖨^c| w_j| d_j/δ_ij^2 + ∑_t∈𝖨, j∈𝖨^c| w_t| d_j/δ_ij^2)  (∑_t∈𝖨,j∈𝖨^c | w_t|d_i/δ_ij) d_i^7 δ_i0Δ(d_i)^3N^-3/2
         
      + (∑_t∈𝖨| w_t| d_i/δ_i0)^2 d_i^6 Δ(d_i)^3N^-3/2.
  
  It is easy to check that (<ref>) is  negligible compared to Λ_2, Λ_3 in def:Lambda_2 and est.S_3. Hence, we can write 
  
      |T_11|=O_≺( Λ_2+Λ_3/N^ε).
   
  
  
  For the term T_12, instead of using the residue theorem as above, we directly bound the integral as follows
  
      |T_12|≤   C∫_Γ_i|z^2(1+z)^2θ(z)^2θ'(z)|  (∑_j| w_j|d_j/|z-d_j|)^2(∑_kd_k/|z-d_k|)^2  |Δ(θ(z))|^4 N^-2
               ×‖ (D^-1+θ(z)V^*_1(θ(z))V)^-1‖_op | dz|.
  
  Notice that for z∈Γ_i,
  
      |z|≍ d_i,    |z-√(y)|≍ d_i-√(y)   and     |z-d_j|≍{[  |d_i-d_j| for  j∈𝖨^c;        ρ_i   for  j∈𝖨 ]. .
  
  Besides,
  
      ‖ (D^-1+θ(z)V^*_1(θ(z))V)^-1‖_op
         =‖( (D^-1+θ(z)m_1(θ(z)))+ θ(z)V^*Ξ(θ(z))V )^-1‖_op
         ≺1/min_j|z-d_j|/ |zd_j| -‖θ(z)V^*Ξ(θ(z))V‖_op,
  
  where for the last bound we used the fact that θ(z)m_1(θ(z))=-(1+1/z). By elementary computations and the definitions of ρ_i, δ_ij (see def:radius and  def:d_i-d_j), one has
  
      min_j∈𝖨^c|z-d_j|/ |z d_j|≍min_j∈𝖨^c|d_i-d_j|/d_id_j≥min_j∈𝖨^cδ_ij/d_i(d_i+δ_ij)≥ρ_i/d_i(d_i+ρ_i)≍ρ_i/d_i^2, 
         min_j∈𝖨 |z-d_j|/|z d_j| ≍ρ_i/d_i^2,
  
  and  
  
      ‖θ(z)V^*Ξ(θ(z))V‖_op   ≺ d_i‖ V^*Ξ(θ(z))V‖_op
         =O_≺ (d_imin{ |θ(z)-λ_+|^-2, |θ(z)-λ_+|^-1/4} N^-1/2)  
         =O_≺ (d_iΔ(d_i)N^-1/2).
  
  Note that the following bounds are implied by the non-overlapping condition asd2
  
      δ_i/d_i^3Δ(d_i)N^-1/2≥ CN^ε,  d_i-√(y)/d_i^3Δ(d_i)N^-1/2≥ CN^ε,
  
  where δ_i=min_j∈𝖨^c|d_i-d_j|. 
   Recall ρ_i from def:radius. Thus
  
      ‖θ(z)V^*Ξ(θ(z))V‖_op≺ N^-ερ_i/d_i^2 .
  
  Consequently, from 2020011301, we get 
  
      ‖ (D^-1+θ(z)V^*_1(θ(z))V)^-1‖_op≺d_i^2/ρ_i.
   
  
  Inserting the estimates 2020011303 and 2020011304 into (<ref>), we arrive at 
  
      |T_12| ≺   (∑_j∈𝖨^c| w_j|d_j/δ_ij+
      ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij)^2
         ×( ∑_j∈𝖨^cd_j/δ_ij+∑_j∈𝖨^cd_i/δ_ij +
      d_i/δ_i0)^2 d_i^7δ_i0 Δ(d_i)^4N^-2.
  
  Further, applying the estimates 2020011405 to the term ∑_j∈𝖨^cd_j/δ_ij+∑_j∈𝖨^cd_i/δ_ij +
  d_i/δ_i0 above, we get 
  
      |T_12| ≺ N^-ε(∑_j∈𝖨^c| w_j|d_j/δ_ij+
      ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij)^2 d_i^3δ_i0 Δ(d_i)^2 N^-1.
  
  
  
  
  Next, we expand the squared term on the RHS of (<ref>) and analyse them term by term.
  Recalling the expression of  Λ_3 in est.S_3, we directly have 
  
      (∑_j∈𝖨^c| w_j|d_j/δ_ij)^2   d_i^3δ_i0 Δ(d_i)^2N^-1≤Λ_3.
  
  Further recalling  the estimates for S_2, i.e. Λ_2 in def:Lambda_2 and using the first estimate in 2020011405, we obtain  
  
      (∑_t∈𝖨| w_t|d_i/δ_i0)^2  d_i^3δ_i0 Δ(d_i)^2 N^-1≺ N^-ε(∑_t∈𝖨| w_t|d_i/δ_i0)^2 δ_i0^2 Δ(d_i) N^-1/2≤ N^-εΛ_2.
  
  It remains to bound the last squared term ( ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij )^2. First notice that 
  it follows from the triangle inequality and d_i/δ_i0≥ 1 that 
  
      ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij≤∑_t∈𝖨,j∈𝖨^c| w_t|d_j/δ_ij+ ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_i0.
  
  By eq:triangle, we have  
  
      (∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij)^2 d_i^3δ_i0 Δ(d_i)^2N^-1
      ≤   (∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij) (∑_t∈𝖨,j∈𝖨^c| w_t|d_j/δ_ij)  d_i^3δ_i0 Δ(d_i)^2N^-1
             +
       (∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij) ( ∑_t∈𝖨| w_t| d_i/δ_i0)  d_i^3δ_i0 Δ(d_i)^2N^-1
      ≤   Λ_3 + Λ_2.
  
  To obtain the last inequality above, we compared the first term with the latter part in the  expression of Λ_3 in est.S_3. And for the second term, we used the non-overlapping condition asd for δ_ij and further compared it with Λ_2 in def:Lambda_2.
  
  
  
  Combining eq:anotherT12-eq:2020040506, we finally have 
  
      |T_12|=O_≺(Λ_2+Λ_3/N^ε)
  
  for some small fixed positive ε.
   
   
   Next, we turn to estimate T_2 and T_3. The method is analogous to that of T_12; we omit the details. By definition, we can bound T_2 and T_3 as follows
   
      |T_2|   =| 1/2π∮_θ(Γ_i)zm_1(z)w^*V( zL(z)V^* Ξ(z)V )^2 (D^-1+zV^*_1(z)V)^-1  V^*Ξ(z) w  dz |
         ≤ C∮_Γ_i| z(1+z)θ(z)^2θ'(z)|  (∑_j| w_j|d_j/|z-d_j|)(∑_kd_k/|z-d_k|) ‖w‖|Δ(z)|^3/ N^3/2
                   ×‖ (D^-1+θ(z)V^*_1(θ(z))V)^-1‖_op | dz| 
         ≺  N^-3/2(∑_j∈𝖨^c| w_j|d_j/δ_ij+
      ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij) ‖w‖
                   ×( ∑_j∈𝖨^cd_j/δ_ij+∑_j∈𝖨^cd_i/δ_ij+
      d_i/δ_i0) d_i^5 δ_i0 Δ(θ(d_i))^3
         ≺ N^-1-ε(∑_j∈𝖨^c| w_j|d_j/δ_ij+
      ∑_t∈𝖨| w_t|d_i/δ_i0+ ∑_t∈𝖨,j∈𝖨^c| w_t|d_i/δ_ij)^2 d_i^3δ_i0 Δ(θ(d_i))^2 
         ≺ N^-ε (Λ_2+Λ_3)
  
  and  
  
      |T_3|   =| -1/2π∮_θ(Γ_i)zw^* Ξ(z)V  ( zL(z)V^* Ξ(z)V )(D^-1+zV^*_1(z)V)^-1  V^*Ξ(z) w  dz |
         ≤ C∮_Γ_i‖w‖^2 | zθ(z)^2θ'(z) | (∑_kd_k/|z-d_k|)  |Δ(z)|^3/ N^3/2‖ (D^-1+θ(z)V^*_1(θ(z))V)^-1‖_op | dz| 
         ≺‖w‖^2  ( ∑_j∈𝖨^cd_j/δ_ij+∑_j∈𝖨^cd_i/δ_ij+
      d_i/δ_i0) d_i^4 δ_i0 Δ(θ(d_i))^3N^-3/2
         ≺ N^-ε (Λ_2+Λ_3).
  
  To obtain the above bounds, we used the estimates in 2020011405 and the  inequality 20200417 for ‖‖.
  Hence, using the estimates of T_1,T_2 and T_3, we get that 
  
      ,  P_𝖨=S_1+S_2+ S_3+O_≺(Λ_2+Λ_3/N^ε),
   
  for some  small  constant ε>0. Finally, using the definitions of Λ_2, Λ_3 in def:Lambda_2 and est.S_3 and the expression of _𝖨, ς_𝖨 in def:v_I_varsigma_I, one can rewrite the error bound 
  
      Λ_2+Λ_3/N^ε   =O_≺( (δ_i0√(d_i) ‖ς_𝖨‖ + d_i‖_𝖨‖) ‖_𝖨‖ Δ(d_i)N^-1/2-ε) 
         +O_≺(  ( d_i‖ς_𝖨‖^2 + d_i^2 ‖w_𝖨‖^2( ∑_j∈𝖨^cd_j/δ_ij) (  ∑_j∈𝖨^cd_i/δ_ij) ) δ_i0  d_i Δ(d_i)^2N^-1-ε)
  
  as in eq:weakgreen. This completes the proof. 
   
  
  Besides the Green representation for the eigenvector, we also have the following lemma  for the representation of the eigenvalue μ_i in the simple case 𝖨={i}. The  extension to multiple case can be found in Appendix <ref>; see Proposition <ref>. 
  
  Suppose that the assumptions in Theorem <ref> hold. In the simple case 𝖨={i},  we have 
  
      μ_i=θ(d_i) - (d_i^2-y) θ(d_i) χ_ii(θ(d_i)) + O_≺( d_i^2δ_i0Δ(d_i)N^-1/2-ε)
  
  for some small fixed constant ε>0.
  
  
  
  
  
  Recall the notations at the beginning of proof of Lemma <ref>, 
      V= (_1,⋯, _r),    D=diag(d_1/(1+d_1), ⋯, d_r/(1+d_r)).
  
   Then, by elementary calculation, we have 
   
      Q-z=Σ^1/2_1^-1(z)(I_M+z_1(z)VDV^*)Σ^1/2.
  
   Notice that μ_i is the ith largest real value such that (Q-μ_i)=0. Further by the fact that μ_i stays away from the spectrum of H with high probability (Lemma  <ref>,  Assumption <ref>), together with the identity (I_M+z_1(z)VDV^*)=(D)(D^-1+zV^*_1(z)V), we have that μ_i is the ith largest real solution to the equation (D^-1+z V^*_1(z)V)=0 with high probability. 
   
  For x∈ [λ_++ N^-2/3+ δ, ∞) ∩(θ(d_i)- d_i^1/2δ_i0^1/2N^-1/2 + δ, θ(d_i)+ d_i^1/2δ_i0^1/2N^-1/2 + δ) with sufficiently small constant δ>0,  we define the matrices 𝒜(x)=(𝒜_ij(x)) and 𝒜(x)=(𝒜_ij(x)) by setting
   
      𝒜_ij(x)=(1+d_i^-1)δ_ij+xv_i^*_1(x)v_j-xm_1(x)δ_ij, 
         𝒜_ij(x)=δ_ij((1+d_i^-1)+x𝐯_i^*_1(x)v_i-xm_1(x)),
  
   Further, we denote the eigenvalues of 𝒜(x) and 𝒜(x) by a_1(x)≤…≤ a_r(x) and a_1(x)≤…≤a_r(x) respectively. Apparently, one has
   
      a_i(x)= (1+d_i^-1)+x𝐯_i^*_1(x)v_i-xm_1(x),
  
   with high probability by the isotropic local law (<ref>) and the Assumption <ref>. We then claim that, in order to prove  (<ref>), it suffices to show  the following two estimates
   
      μ_im_1(μ_i)=-a_i(θ(d_i))+ O_≺( d_i^-1/2δ_i0^-1/2 N^-1/2- ε),
          a_i(θ(d_i))=a_i(θ(d_i))+ O_≺( d_i^-1/2δ_i0^-1/2 N^-1/2- ε).
  
   for some small constant ε>0.
   Combining (<ref>), (<ref>) and (<ref>), we have 
   
      μ_im_1(μ_i)- θ(d_i)m_1(θ(d_i))
           = -(1+d_i^-1)-θ(d_i)𝐯_i^*_1(θ(d_i))v_i + O_≺( d_i^-1/2δ_i0^-1/2 N^-1/2- ε).
  
   Expanding μ_im_1(μ_i) around θ(d_i)m_1(θ(d_i)) with the aid of Lemma <ref>  will then lead to  (<ref>). Therefore, what remains is to prove (<ref>) and (<ref>).  
   
   We start with (<ref>).  First, by the fact that μ_i is a solution to (D^-1+z V^*_1(z)V)=0, it is easy to see that 
   μ_im_1(μ_i)= -a_k(μ_i) for some k.  But by isotropic law (<ref>),  we see that  𝒜=𝒜+O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2), where O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2) represents a matrix bounded in  operator norm  by O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2). This leads to the  estimate a_k(μ_i)=a_k(μ_i)+O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2). Further, by  (<ref>) and Lemma <ref> one can easily show that μ_im_1(μ_i)=-(1+d_i^-1)+O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2) and a_k(μ_i)=(1+d_k^-1)+O_≺(d_i^-1/2δ_i0^-1/2 N^-1/2).  Therefore, due to the fact that d_i's are well separated, more specifically, non-overlapping condition asd2, we have μ_im_1(μ_i)= -a_i(μ_i)  with high probability. Next, by the isotropic law (<ref>), one can also check that 
   
      ∂_x 𝒜(x)_op≺ d_i^1/2δ_i0^-5/2 N^-1/2
  
   This together with Lemma <ref> leads to
   
      |a_i(μ_i)-a_i(θ(d_i))|≺  d_i δ_i0^-2 N^-1≺ d_i^-1/2δ_i0^-1/2N^-1/2-ε,
  
   where the last step is due to the fact that  d_iδ_i0^-3/2 N^-1/2 = O(N^-ε) following from asd.
   
   Combining the above with the fact μ_im_1(μ_i)= -a_i(μ_i)  with high probability, we arrive at (<ref>). 
   
    Next, we prove (<ref>).  Observe that the diagonal entries of 𝒜-𝒜 are 0, and 𝒜 is a diagonal matrix. So expanding the eigenvalues of 𝒜 around the eigenvalues of 𝒜 using the perturbation theory, we see that the first order  term vanishes. Hence, it suffices to estimate the second order term. More specifically, we have 
   
      |a_i(θ(d_i))-a_i(θ(d_i))|≺𝒜-𝒜_op^2/min_j≠ i |a_j(θ(d_i))-a_i(θ(d_i))|≺ d_i^-1/2δ_i0^-1/2 N^-1/2- ε,
   
  for some small fixed ε>0, where the last step follows from the fact that a_k(θ(d_i))=1+d_k^-1+O_≺(d_i^-1/2δ_i0^-1/2N^-1/2) and the fact that the d_i's are well separated and satisfy asd2. This concludes the proof of (<ref>).
  
  
  
  
  
  
  
  
  § PROOF OF THEOREMS <REF> AND <REF> 
   
  
  
  
  In this section, we prove the  main result Theorem <ref>, based on Proposition <ref>. The proof of Theorem <ref> is nearly the same, and thus it will be  only briefly stated in the end of the section. The proof of Proposition <ref> will be deferred to Section <ref>. The starting point  is Lemma <ref> and  Remark <ref>, which state that  the study of ,  P_𝖨 can be  reduced to the study of the random vector 
  
      [        _𝖨^*Ξ' (z)_𝖨,         _𝖨^*Ξ (z)_𝖨,        ς_𝖨^*Ξ(z) _𝖨,  {_t^*Ξ (z)ς_𝖨}_t∈𝖨, {_j^* Ξ (z)_𝖨}_j∈𝖨^c ]^*
  
  at  z=θ(d_i). 
   
    
  In view of the isotropic local laws in Theorem <ref> and Lemma <ref>, we shall study the limiting distribution of the rescaled random vectors 
   
      χ_𝖨(z):= /Δ(d_i)( c_1 (_𝖨^0)^*Ξ(z)_𝖨^0+ c_2(_𝖨^0)^*Ξ'(z)_𝖨^0,  (ς_𝖨^0)^*Ξ(z)_𝖨^0, {_t^*Ξ  (z)ς_𝖨^0}_t∈𝖨, {_j^*Ξ (z)_𝖨^0}_j∈𝖨^c)
  
  at z=θ(d_i), where  _𝖨^0 and ς_𝖨^0 are defined  in (<ref>) and (<ref>) and we further introduced the shorthand notations
   
      c_1≡ c_1(d_i):= 2d_i   and   c_2≡ c_2(d_i):= (d_i^2-y)^2/d_i^2.
  
  We remark here, in the case when d_i>K for sufficiently large K or even grows with N, that either summand in the first component of χ_𝖨(z) may have larger typical order than the other components of the vector χ_𝖨(z), by a direct application of the isotropic local laws weak_est_DG. However, for such large d_i, referring to  the estimate of  weak_19071941  in the proof of Lemma <ref>, more precisely, cancel:1-2020072106, we shall also exploit a cancellation between two summands in the first component of (<ref>) that is simply proportional to the combination in weak_19071941.
  Consequently, all the components of χ_𝖨(z) are of comparable sizes. The details of such arguments will be provided in later proofs. Thus our goal is to prove that χ_𝖨(z) is asymptotically Gaussian (c.f. Lemma <ref>) and this leads to the proof of Theorem <ref>.
  
  Before we give the precise statement, let us introduce some necessary notations. For brevity, in the sequel, we will very often omit the z-dependence from the notations. For instance, we will write m_1,2(z) as m_1,2. 
  
  In the sequel, we fix an i∈ 1, r_0 and its corresponding index set 𝖨≡𝖨(i). For simplicity, we also denote
  
      c_1= d_i^2+2d_iy+y/d_i+y,   c_2= c_2= (d_i^2-y)^2/d_i^2.
  
  Define a symmetric matrix  ℳ_𝖨≡ℳ_𝖨(z)∈ℂ^(r+2)× (r+2) with diagonal entries 
  
      ℳ_𝖨 (j,j) = 
      2/Δ(d_i)^2 ∑_a,b=1^2  c_a  c_bm_1 ∑_c
      a_1,a_2≥ 1
       a_1+a_2=a+1
      m_1^(a_1-1) (zm_1)^(b+a_2-1)/(a_1-1)!(b+a_2-1)!,
          if  j=1;
      
      1/Δ(d_i)^2 m_1^2(zm_1)',     otherwise  ;
  
   and its non-zero off-diagonal  entries as
   
      ℳ_𝖨(2,  3,r+2 )=
      1/Δ(d_i)^2(m_1^2(zm_1)') ( {_𝖨^0, _t}_t∈𝖨 ,  {ς_𝖨^0, _j}_j∈𝖨^c),
         ℳ_𝖨( 3,|𝖨|+ 2 ,  |𝖨|+3,r+2 )=1/Δ(d_i)^2(m_1^2(zm_1)') (  _𝖨^0, _tς_𝖨^0, _j)_t∈𝖨,j∈𝖨^c.
  
  Here  for any matrix A=(a_ij)_n× n and  index sets 𝒮_1,𝒮_2⊂1, n,   we denote by A(𝒮_1,𝒮_2) the submatrix of A obtained via taking rows from 𝒮_1 and columns from 𝒮_2. More specifically, 
  
      A(𝒮_1,𝒮_2):=(a_ij)_i∈𝒮_1, j∈𝒮_2.
  
  When 𝒮_1={a} is a singleton, we use the abbreviation A(a, 𝒮_2) instead of A({a}, 𝒮_2). 
  We then further define the  symmetric matrix  𝒦_𝖨≡𝒦_𝖨(z)∈ℂ^(r+2)× (r+2) whose 
  r × r lower right corner has the block structure 
  
      (
      [      𝒦_𝖨(𝖨,𝖨)   𝒦_𝖨(𝖨, 𝖨^c);   𝒦_𝖨(𝖨^c, 𝖨) 𝒦_𝖨(𝖨^c, 𝖨^c) ]),
  
  given by 
  
      𝒦_𝖨(𝖨, 𝖨)= 1/Δ(d_i)^2(zm_2m_1^2)^2 (𝐬_1,1,2(_t_1,_t_2,ς_𝖨^0))_t_1,t_2∈𝖨, 
         𝒦_𝖨(𝖨^c, 𝖨^c)= 1/Δ(d_i)^2 (zm_2m_1^2)^2 (𝐬_1,1,2(_j,_k,_𝖨^0))_j,k ∈𝖨^c,
         𝒦_𝖨(𝖨, 𝖨^c)= 1/Δ(d_i)^2 (zm_2m_1^2)^2 (𝐬_1,1,1,1(_t,_j, ς_𝖨^0,_𝖨^0))_t∈𝖨, j∈𝖨^c.
  
  The remaining  entries  are defined by 
   
      𝒦_𝖨(1,1)=   𝐬_4(_𝖨^0)/Δ(d_i)^2(∑_s=1^2  c_sm_1(zm_2m_1)^(s-1)/(s-1)!)( ∑_s=1^2  c_s(zm_2m_1^2)^(s-1)/(s-1)!),
      
         𝒦_𝖨(1,  2, r+2 )= zm_2m_1^2/2Δ(d_i)^2(∑_s=1^2  c_sm_1(zm_2m_1)^(s-1)/(s-1)! +  ∑_s=1^2  c_s(zm_2m_1^2)^(s-1)/(s-1)!) 
                         × (𝐬_1,3(ς_𝖨^0, _𝖨^0), {𝐬_1,1,2(_t , ς_𝖨^0 ,_𝖨^0 )}_t∈𝖨, {𝐬_1,3(_j, _𝖨^0)}_j∈𝖨^c), 
         𝒦_𝖨(2, 2, r+2)=(zm_2m_1^2)^2/Δ(d_i)^2
                         ×(𝐬_2,2(ς_𝖨^0,_𝖨^0), {𝐬_1,1,2(_t,_𝖨^0 , ς_𝖨^0)}_t∈𝖨, {𝐬_1,1,2(_j, ς_𝖨^0 , _𝖨^0)}_j∈𝖨^c) .
  
  Finally, we  define 
  
      𝒱_𝖨(z)=ℳ_𝖨(z)+κ_4 𝒦_𝖨(z).
  
  
  
  Recall χ_𝖨 defined in (<ref>).  We have the following lemma. 
   Under the assumptions of Theorem <ref>, we have
  
      χ_𝖨(θ(d_i)) ≃𝒩(0, 𝒱_𝖨(θ(d_i)) ).
  
  
  
  
  With the above lemma, we now can finish the proof of Theorem <ref>.
  
  Recall the shorthand notation δ_i0=d_i-√(y). Set
   
      Θ_𝖨^:=- (d_i^2- y)^1/2(1+d_i)Δ(d_i)(  /Δ(d_i)( c_1 (_𝖨^0)^*Ξ(z)_𝖨^0+ c_2(_𝖨^0)^*Ξ'(z)_𝖨^0)),
      
         Λ_𝖨^:=-2(d_i^2- y)√(d_i+1/δ_i0) Δ(d_i) (/Δ(d_i)(ς_𝖨^0)^*Ξ(θ(d_i))_𝖨^0),
         Δ_𝖨t^:=-√(d_ig(d_i)) Δ(d_i)  (/Δ(d_i)_t^*Ξ(θ(d_i))ς_𝖨^0)    for  t∈𝖨,
         Π_𝖨j^:=-√((1+d_i)g(d_i)) Δ(d_i)  (/Δ(d_i)_j^*Ξ(θ(d_i))_𝖨^0)    for  j∈𝖨^c.
  
  Notice that Θ_𝖨^,  Λ_𝖨^, Δ_𝖨t^ and Π_𝖨j^ for t∈𝖨, j∈𝖨^c are all linear combinations of the components of χ_𝖨(θ(d_i)). Then by Lemma <ref>, they are also asymptotically jointly Gaussian with mean 0. The entries of the covariance matrix of (Θ_𝖨^,  Λ_𝖨^, {Δ_𝖨t^}_t∈𝖨, {Π_𝖨j^}_j∈𝖨^c) can be obtained from 𝒱_𝖨(θ(d_i)). Further simplifications can be achieved using the following identities at z=θ(d_i)
  
      m_1^2(zm_1)' = 1/(d_i+y)^2(d_i^2-y),      m_1^2(zm_1)”= -2d_i^3/(d_i+y)^2(d_i^2-y)^3, 
          m_1^2(zm_1)”'= 6d_i^4(d_i^2+y)/(d_i+y)^2(d_i^2-y)^5,      m_1m_1'(zm_1)'= - d_i^2/(d_i+y)^3(d_i^2-y)^2 ,
         m_1m_1'(zm_1)” = 2d_i^5/(d_i+y)^3(d_i^2-y)^4,      zm_2m_1^2=-1/d_i(d_i+y),
      
         (zm_2m_1^2)'=2d_i+y/(d_i+y)^2(d_i^2-y),      m_1(zm_2m_1)'= 1/(d_i+y)(d_i^2-y),
  
  as well as the explicit expressions for c_1, c_2,  c_1,  c_2 in def:c12 and def:wtc12. 
  
  We further set 
   
      Θ__𝖨^=Θ_𝖨^_𝖨^2,   Λ_ς_𝖨^=Λ_𝖨^_𝖨,   Δ__t^=Δ_𝖨t^,   Π__j^= Π_𝖨j^_𝖨.
   
   It follows immediately from Lemma <ref> that , P_𝖨 can be written as the RHS of weak_gf_decomp in  Theorem <ref> and 
   
      ( Θ__𝖨^, Λ_ς_𝖨^, {Δ__t^}_t∈𝖨, {Π__j^}_j∈𝖨^c) ≃𝒩(0, A_𝖨^ + κ_4  d_i^2-y/d_i^2 B_𝖨^),
   
   with the matrices A_𝖨^ and B_𝖨^ defined in (<ref>) and (<ref>). This completes the proof of Theorem <ref>.
  
  
  
  
  In the rest of this section, we prove Lemma <ref>, based on our key technical result, Proposition <ref>. In order to show the asymptotic Gaussianity of χ_𝖨(θ(d_i)), it suffices to show that all  linear combinations of the components of χ_𝖨(θ(d_i)) are asymptotic Gaussian.  Our proof will be based on a moment estimate. This requires a deterministic bound for the Green function,  in order to control the contribution of the bad event in the isotropic local laws. To this end, we introduce a tiny imaginary part to the parameter z, such that the Green functions can be bounded by 1/ z deterministically. Specifically, in the sequel, we set 
  
      z=θ(d_i)+ N^-K,
  
  for some sufficiently large constant K>0. 
  
  For a fixed deterministic  column vector c=(c_01,   c_02, {c_1t}_t∈𝖨,  {c_2j}_j∈𝖨^c )^*∈ℝ^r+2, we define
  
      :=c^*χ_𝖨(z),
  
  where z is given in (<ref>).  Notice that |𝒫|≺ 1 by the isotropic local law. 
   Here we omit the dependence of 𝒫 on 𝐜 and the index set 𝖨 for simplicity. Hereafter we always assume that 𝐜 and 𝖨 are fixed. 
  The following proposition is our main technical task. 
  
    Let 𝒫 be defined in (<ref>) with z given in (<ref>). Under the assumption of  Theorem <ref>, we have 
  
      (i)   𝒫=O_≺(N^-1/2δ_i0^-3/2d_i^3/2), 
         (ii)  𝒫^l =(l-1)𝒱_𝖨,𝐜𝒫^l-2+O_≺(N^-1/2δ_i0^-3/2d_i^3/2),
  
  where 𝒱_𝖨,𝐜=c^*𝒱_𝖨 (θ(d_i))c with  𝒱_𝖨(θ(d_i)) defined in (<ref>) and δ_i0= d_i-√(y) is defined in  def:d_i-d_j.
  
  
  With the above proposition, we can now show the proof of Lemma <ref>. 
  
  
  By Proposition <ref>, one observes that (z) is asymptotically Gaussian with mean 0 and variance 𝒱_𝖨,𝐜. By the definition of z in (<ref>), and a simple continuity argument for the Green function, one can easily see that 𝒫(θ(d_i)) admits the same asymptotic distribution as  (z), when K is chosen to be sufficiently large. 
  Since  𝒱_𝖨,𝐜=c^*𝒱_𝖨(θ(d_i))c and c is arbitrary, we have 
  
      χ_𝖨(θ(d_i)) ≃𝒩(0, 𝒱_𝖨(θ(d_i)) ).
  
  This concludes the proof of Lemma <ref>. 
  
  
  
  
  In the end of this section, we briefly state the proof of Theorem <ref>.
   
   In addition to the expansion of ,  P_i which can be easily obtained from Lemma <ref> in case 𝖨={i}, by setting 
   
      Φ_i:= - √(d_i^2-y)θ(d_i)χ_ii(θ(d_i)),
   
   it follows immediately from Lemma <ref> that 
   the  eigenvalue μ_i can be written in terms of Φ_i. Note that Φ_i is also a quadratic form similar to the other quadratic forms in   the expansion of eigenvector in Lemma  <ref> in case  𝖨={i}. Hence their joint distribution can de deduced analogously to the proof of Theorem <ref> by adding one more quadratic form into the linear combinations of quadratic forms 𝒫. Since the proof strategy of Theorem <ref> can apply mutatis mutandis to that of Theorem  <ref>, we omit the details and conclude the proof. 
   
  
  
  Acknowledgment. The second author would like to thank Igor Silin for sharing the Python codes of <cit.> and providing some insights on the statistical applications. 
  
  
  
  
  
  §  COLLECTION OF DERIVATIVES
   
  
  In this section, we summarize some derivatives that appear in the previous sections. And all these derivatives can be obtained by repeatedly applying the second identity in derivative and chain rule. 
  For convenience, we set 
  
      𝒪_l:={(o_1,⋯,o_l):∃ i∈{1,⋯,l}, o_i=0, and o_j=1,2   for all j≠ i}
  
  
  
  
  For simplicity, in the sequel,  we still use _a^s, _a^(s) to denote Υ^s-1_a^s, Υ^s_a^(s) respectively  for a=1,2. Below,  we first collect  the derivatives of (X^*_1^s)_k  for any deterministic unit vector , 
  which can be derived by using (<ref>) and the product rule.
  The first derivative of (X^*_1^s)_k is
  
      ∂ (X^*_1^s)_k/∂ x_qk=(_1^s)_q-∑_a=1^2∑_cs_1,s_2≥ 1;
       s_1+s_2=s+1 (X^*_1^s_1𝒫_a^qk_1^s_2)_k.
  
  The second derivative of  (X^*_1^s)_k is
  
      ∂^2 (X^*_1^s)_k/∂ x_qk^2=   2(-∑_a=1^2∑_cs_1,s_2≥ 1;
       s_1+s_2=s+1 (_1^s_1𝒫_a^qk_1^s_2)_q
         +∑_a_1,a_2=1^2∑_cs_1, s_2, s_3≥ 1;
      ∑_i=1^3s_i=s+2 (X^*_1^s_1𝒫_a_1^qk_1^s_2𝒫_a_2^qk_1^s_3)_k
         
      -∑_cs_1,s_2≥ 1;
       s_1+s_2=s+1 (X^*_1^s_1𝒫_0^qk_1^s_2)_k).
  
  The third derivative of  (X^*_1^s)_k is
  
      ∂^3 (X^*_1^s)_k/∂ x_qk^3=    6(∑_a_1,a_2=1^2∑_cs_1,s_2,s_3≥ 1;
      ∑_i=1^3s_i=s+2 (_1^s_1𝒫_a_1^qk_1^s_2𝒫_a_2^qk_1^s_3)_q
         -∑_cs_1,s_2≥ 1;
       s_1+s_2=s+1 (_1^s_1𝒫_0^qk_1^s_2)_q
         -∑_a_1,a_2,a_3=1^2∑_cs_1,⋯,s_4≥ 1;
      ∑_i=1^4s_i=s+3 (X^*(∏_i=1^3(_1^s_i𝒫_a_i^qk))_1^s_4)_k
         +∑_(a_1,a_2)∈𝒪_2∑_cs_1,⋯,s_3≥ 1;
      ∑_i=1^3s_i=s+2 (X^*(∏_i=1^2(_1^s_i𝒫_a_i^qk))_1^s_3)_k).
  
  The fourth derivative of  (X^*_1^s)_k is
  
      ∂^4 (X^*_1^s)_k/∂ x_ik^4=   4!(-∑_a_1,a_2,a_3=1^2∑_cs_1,⋯,s_4≥ 1;
      ∑_i=1^4s_i=s+3 ((∏_i=1^3(_1^s_i𝒫_a_i^qk))_1^s_4)_q
         +∑_(a_1,a_2)∈𝒪_2∑_cs_1,s_2,s_3≥ 1;
      ∑_i=1^3s_i=s+2 ((∏_i=1^2(_1^s_i𝒫_a_i^qk))_1^s_3)_q
         +∑_a_1,⋯,a_4=1^2∑_cs_1,⋯,s_5≥ 1;
      ∑_i=1^5s_i=s+4(X^*
      (∏_i=1^4(_1^s_i𝒫_a_i^qk))_1^s_5)_k
         -∑_(a_1,a_2,a_3)∈𝒪_3∑_cs_1,⋯,s_4≥ 1;
      ∑_i=1^4s_i=s+3 (X^*(∏_i=1^3(_1^s_i𝒫_a_i^qk))_1^s_4)_k
         +∑_cs_1,s_2,s_3≥ 1;
      ∑_i=1^3s_i=s+2 (X^*(∏_i=1^2(_1^s_i𝒫_0^qk))_1^s_3)_k).
  
  
  
  Next, for 𝒫 defined in (<ref>), we collect its derivatives  
      ∂^s /∂ x_qk^s=/Δ(d_i)y_0^* (∑_a=1^2 c_a
      ∂^s _1^a/∂ x_qk^s) ϑ_0 + /Δ(d_i)∑_t=1^2   y_t^* ∂^s _1/∂ x_qk^sϑ_t.
   For the first derivative, we have
  
      ∂/∂ x_qk   =-/Δ(d_i)∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ((y_0^*_1^a_1)_q(X^*_1^a_2ϑ_0)_k+(X^*_1^a_1y_0)_k(_1 ^a_2ϑ_0)_q)
            - /Δ(d_i)∑_t=1^2 ((y_t^*_1)_q(X^*_1ϑ_t)_k+(X^*_1y_t)_k(_1ϑ_t)_q).
  
  By weak_est_XG and Remark <ref>, it is easy to see that the second term on the RHS of eq:Pd1 can be estimated by 
  
      | /Δ(d_i)∑_t=1^2 ((y_t^*_1)_q(X^*_1ϑ_t)_k+(X^*_1y_t)_k(_1ϑ_t)_q) |    ≺Δ(d_i)^-1d_i^-3/2(d_i-√(y))^-1/2
         ≍ 1,
  
  where the last step follows from the estimate of Δ(d_i) in Delta:asymp. For the first term on the RHS of  eq:Pd1, first using  weak_est_DG, We see
  that 
  
      ∂/∂ x_qk   = - /Δ(d_i)[ 
      (c_1/2 m_1+ c_2 m_1')(X^*_1ϑ_0)_k y_0q
         + (c_1/2 (X^*_1ϑ_0)_k + c_2  (X^*_1^2ϑ_0)_k )(_1^*y_0)_q+ (c_1/2 m_1+ c_2 m_1') (X^*_1y_0)_k ϑ_0q
         
      + (c_1/2 (X^*_1y_0)_k + c_2  (X^*_1^2y_0)_k ) (_1^*ϑ_0)_q
      ] +O_≺(d_i^1/2δ_i0^-1/2N^-1/2).
  
  Note that by plugging the values of c_1,c_2 and m_1, m_1' at z=θ(d_i), we have
  
      c_1/2 m_1+ c_2 m_1'=-y(d_i+1)/(d_i+y)^2 =O(d_i^-1).
  
  If d_i is large, i.e. d_i>K for sufficiently large constant K>0, performing the expansions for _1, _1^2 around  -1/θ(d_i) and 1/(θ(d_i))^2 respectively, we can further see the cancellation
  
      c_1/2 (X^*_1ϑ_0)_k + c_2  (X^*_1^2ϑ_0)_k
          =O(d_i) ×( (X^*_1ϑ_0)_k  +θ(d_i) (X^*_1^2ϑ_0)_k)+O_≺(N^-1/2) 
         = O(d_i) × O_≺( (X^*Hϑ_0)_k/θ(d_i)^2) = O_≺(d_i^-1N^-1/2).
  
  which leads to ∂/∂ x_qk=O_≺(1). Here we used  the estimate 
  (X^*Hϑ_0)_k =O_≺(N^-1/2), which can be easily verified by the moment method.
   For the case d_i=O(1), simply using isotropic local laws weak_est_XG and  Remark <ref>, we  obtain ∂/∂ x_qk=O_≺(1). To conclude, we always have
  
      ∂/∂ x_qk=O_≺(1).
  
  
  The second derivative of  is
  
      ∂^2 /∂ x_qk^2   =2/Δ(d_i)∑_b=1^2 c_b (∑_a_1,a_2=1^2∑_cb_1,b_2,b_3≥ 1;
      ∑_i=1^3b_i=b+2 y_0^*(∏_j=1^2(_1^b_j𝒫_a_j^qk))_1^b_3ϑ_0 
                      -∑_cb_1,b_2≥ 1;
       b_1+b_2=b+1 y_0^*_1^b_1𝒫_0^qk_1^b_2ϑ_0)
         +2/Δ(d_i)∑_t=1^2  (∑_a_1,a_2=1^2 y_t^*(∏_j=1^2(_1𝒫_a_j^qk))_1ϑ_t - 
       y_t^*_1𝒫_0^qk_1ϑ_t).
  
  Recalling the definition of 𝒫^qk_i for i=0,1,2 in P012, one observes that all the terms in the parenthesis admit  one of the following forms 
  
      (X^*_1^b_1η_1)_k(X^*_1^b_2)_kq(_1^b_3η_2)_q,      (X^*_1^b_1η_1)_k(_1^b_2)_qq(X^*_1^b_3η_2)_k, 
         
      (_1^b_1η_1)_q(X^*_1^b_2X)_kk^a(_1^b_3η_2)_q
  
  for η_1,η_2=y_0, y_1,y_2, ϑ_0, ϑ_1, ϑ_2, a=0,1 and b_1,b_2,b_3=1,2 satisfying 
  b_1+b_3-1+a(b_2-1)=b ∈{1,2}, which are bounded by O_≺ (N^-1(d_i-√(y))^-1d_i^-2), O_≺ (N^-1 (d_i-√(y))^-1d_i^-2), O_≺( d_i^-2) respectively for b=1 and by O_≺ (N^-1(d_i-√(y))^-3d_i^-1), O_≺ (N^-1 (d_i-√(y))^-3d_i^-1), O_≺( (d_i-√(y))^-1d_i^-2) respectively for b=2, in light of weak_est_XG and Remark <ref>. Therefore, combining with the prefactor  and coefficients c_1,2 in (<ref>), we get 
  
      ∂^2 /∂ x_qk^2   = 2/Δ(d_i)[
       c_1 ((_1y_0)_q(X^*_1X)_kk(_1ϑ_0)_q
       - (_1y_0)_q(_1ϑ_0)_q) 
            + c_2 ( (_1^2y_0)_q(X^*_1X)_kk(_1 ϑ_0)_q + 
       (_1y_0)_q(X^*_1^2X)_kk(_1ϑ_0)_q
           
       +(_1y_0)_q(X^*_1X)_kk(_1^2ϑ_0)_q - (_1^2y_0)_q(_1ϑ_0)_q- (_1y_0)_q(_1^2ϑ_0)_q)
       ]  
           
       + O_≺ (Δ(d_i)^-1 d_i^-2)
         = 2/Δ(d_i)[
       c_1 ((_1y_0)_q(z_2)_kk(_1ϑ_0)_q
       ) + c_2 ( (_1^2y_0)_q(z_2)_kk(_1 ϑ_0)_q
            + 
       (_1y_0)_q(_2+z_2^2)_kk(_1ϑ_0)_q   +(_1y_0)_q(z_2)_kk(_1^2ϑ_0)_q) ]  
           
       + O_≺ ((d_i-√(y))^1/2 d_i^-1/2)
         = O_≺( 2/Δ(d_i)(c_1m_1^2(zm_2)  + c_2(2m_1'm_1(zm_2)+ m_1^2(zm_2)')) ) 
            + O_≺ ((d_i-√(y))^1/2 d_i^-1/2) 
          = O_≺ ((d_i-√(y))^1/2 d_i^-1/2) .
  
  In the last step above, we used
  
      c_1m_1^2(zm_2)  + c_2(2m_1'm_1(zm_2)+ m_1^2(zm_2)')=-y(d_i^2+2d_i+y)/d_i^2(d_i+y)^2= O(d_i^-2).
  
  
  The third derivative of  is
  
      ∂^3 /∂ x_qk^3   =-6/Δ(d_i)∑_b=1^2 c_b  (∑_a_1,a_2,a_3=1^2∑_cb_1,⋯,b_4≥ 1;
      ∑_i=1^4 b_i=b+3 (y_0^*(∏_j=1^3(_1^b_j𝒫_a_j^qk))_1^b_4ϑ_0)
             -∑_(a_1,a_2)∈𝒪_2∑_cb_1,b_2,b_3≥ 1;
      ∑_i=1^3b_i=b+2 (y_0^*(∏_j=1^2(_1^b_j𝒫_a_j^qk))_1^b_3ϑ_0)) 
            -6/Δ(d_i)∑_t=1^2  (∑_a_1,a_2,a_3=1^2(y_t^*(∏_j=1^3(_1𝒫_a_j^qk))_1 ϑ_t) 
             
      -∑_(a_1,a_2)∈𝒪_2(y_t^*(∏_j=1^2(_1𝒫_a_j^qk))_1ϑ_t)).
  
  By plugging in 𝒫^qk_a defined in (<ref>), one can see that all summands  above contain either more than two quadratic forms of  (X^*_1^a)  and  one quadratic form of (_1^b), or one  quadratic form of  (X^*_1^a)  and  two quadratic forms of (_1^b) for some a, b=1, 2,  which  by weak_est_XG and weak_est_DG will contribute a O_≺ (N^-1/2 d_i^-1/2+(a-1)(d_i-√(y))^-1/2-2(a-1)) factor and a O_≺(d_i^-1(d_i-√(y))^-(a-1)) factor respectively. Using this fact together with contribution of c_b/Δ(d_i), one can easily obtain the crude bound 
  
      |∂^3 /∂ x_qk^3|≺ 1.
  
  
  
  The fourth derivative of  is
  
      ∂^4 /∂ x_qk^4   =4!/Δ(d_i)∑_b=1^2 c_b [∑_a_1,⋯,a_4=1^2∑_cb_1,⋯,b_5≥ 1;
      ∑_i=1^5 b_i=b+4 (y_b^*
      (∏_j=1^4(_1^b_j𝒫_a_j^qk))_1^b_5ϑ_0)
             -∑_(a_1,a_2,a_3)∈𝒪_3∑_cb_1,⋯,b_4≥ 1;
      ∑_i=1^4 b_i=b+3 (y_0^*(∏_j=1^3(_1^b_j𝒫_a_j^qk))_1^b_4ϑ_0)
              +∑_cb_1,b_2,b_3≥ 1;
      ∑_i=1^3b_i=b+2 (y_0^*(∏_j=1^2(_1^b_j𝒫_0^qk))_1^b_3ϑ_0)] 
            +4!/Δ(d_i)∑_t=1^2[ ∑_a_1,⋯,a_4=1^2(y_t^*
      (∏_j=1^4(_1𝒫_a_j^qk))_1ϑ_t)
             
      -∑_(a_1,a_2,a_3)∈𝒪_3(y_t^*(∏_j=1^3(_1𝒫_a_j^qk))_1ϑ_t)+(y_t^*(∏_j=1^2(_1𝒫_0^qk))_1ϑ_t) ].
  
   
   
  
  
  
  § PROOF OF PROPOSITION <REF>
   
  
  
  This section is devoted to the proof of Proposition <ref>, the recursive moment estimates of 𝒫 defined in (<ref>). The basic strategy is to use the cumulant expansion formula in Lemma <ref> to the functionals of Green functions. In the context of Random Matrix Theory, such an idea dates back to <cit.>. We also refer to <cit.> for some recent applications of this strategy for other problems in Random Matrix Theory. 
  
  First,  we provide estimates for some random terms that appear frequently in the proof. Indeed, they have  one of the following forms: η_1^*X^*_1^sη_2, η_1^*X^*_1^sXη_2, η_1^*_1^sη_2, for any fixed s∈ℕ and deterministic vectors η_1, η_2 whose ℓ^2-norm are bounded by some constant C>0. Notice that under the choice of z in (<ref>),
  we have the deterministic bound
   
      |η_1^*_1^sη_2|≤ C( z)^-s.
  
  Similarly, by Cauchy-Schwarz inequality, we have 
   
      |η_1^*X^*_1^sη_2|   ≤η_1X^*_1^sη_2≤ C  (η_2^*_1^s(z̅)XX^*_1^s(z)η_2 )^1/2
         = (η_2^*_1^s(z̅)(I+z_1)_1^s-1(z)η_2 )^1/2≤ C( z)^-s
  
   and 
    
      |η_1^*X^*_1^sXη_2|= |η_1^*X^*X_2^sη_2|= |η_1^*_2^s-1η_2+zη_1^*_2^sη_2|≤ C( z)^-s.
  
  According to Lemma <ref> (ii), the above deterministic bounds allow us to use the high probability bounds  for the aforementioned quantities (following from the isotropic local law) directly in the calculation of the expectations in (<ref>) and (<ref>).  
   
  
  
  Next, we derive an initial bound on :=c^*χ_𝖨(z) in 19071950, where χ_𝖨 is the random vector defined in def:chi_I and c=(c_01,   c_02,  {c_1t}_t∈𝖨,  {c_2j}_j∈𝖨^c )^*∈ℝ^r+2 is an arbitrary fixed deterministic column vector of size r+2. For the case d_i≤ K for sufficiently large constant K>0, one can easily get ||=O_≺(1) by a direct application of the isotropic local law weak_est_DG. As for the case d_i>K for sufficiently large constant K>0, it is easy to see that all the components of χ_𝖨(z) other than the first one are of  size O_≺(1) by applying the isotropic local law directly. Actually, as we mentioned earlier, there is a cancellation between two summands in the first component of χ_𝖨(z). After exploiting this cancellation, similarly to the discussion in cancel:1-2020072106, we can also get an O_≺(1) bound for the first component of χ_𝖨(z).
  Hence, according to the above discussion, we  have 
  
      =O_≺(1).
  
  
  
  
  With the above preliminary bounds at hand, we can now start formally the proof of Proposition <ref>. 
  The main tool for the proof is the cumulant expansion formula in Lemma <ref>. For convenience, we first introduce the following three column vectors
   
      y_0:= c_01_𝖨^0,    y_1:= ∑_t∈𝖨 c_1t_t ,   y_2:=c_02ς_𝖨^0+  ∑_j∈𝖨^cc_2j_j
  
   and define 
      ϑ_0=ϑ_2=_𝖨^0,  ϑ_1=ς_𝖨^0.
  
  Then we can rewrite
  
      =/Δ(d_i)y_0^*( ∑_a=1,2 c_a(_1^(a-1)-m_1^(a-1)))ϑ_0
      +
      /Δ(d_i)∑_t=1,2y_t^*(_1-m_1)ϑ_t
  
  and 
  
      𝔼(^l)
      =/Δ(d_i)𝔼( y_0^*( ∑_a=1,2 c_a(_1^(a-1)-m_1^(a-1)))ϑ_0
      +
      ∑_t=1,2y_t^*(_1-m_1)ϑ_t)
       ^l-1.
  
  Using the identity
  
      _1^t=z^-1(H_1^t-_1^t-1),      t=1,2,
  
  we  rewrite  (<ref>) as
  
      𝔼(^l)=   /Δ(d_i)𝔼(
      y_0^* ( c_2(1/(1+m_2)z H_1^2 + m_2/1+m_2_1^2 +(zm_2)'/(1+m_2)z_1 -(zm_2)' + 1 /(1+m_2)z_1 -m_1')
         + c_1 ( 1/(1+m_2)z H_1 + m_2/1+m_2_1-  1/(1+m_2)z -m_1 )
      ) ϑ_0
         + ∑_t=1,2y_t^*   ( 1/(1+m_2)z H_1 + m_2/1+m_2_1-  1/(1+m_2)z -m_1 )  ϑ_t)
      ^l-1.
  
  Repeatedly using the first and third identities in identitym1m2, we have
  
      𝔼(^l)=   -  m_1/Δ(d_i)𝔼(
      y_0^* ( c_2( H_1^2 + zm_2 _1^2 +(zm_2)'_1 )
         + c_1 (  H_1 + zm_2 _1 ) - c_2 m_1'/m_1^2 (_1-m_1)
      ) ϑ_0
         + ∑_t=1,2y_t^*   (  H_1 +zm_2 _1 )  ϑ_t)
      ^l-1,
  
  which can be further simplified to 
  
      𝔼(^l)=   -  m_1/Δ(d_i)𝔼(
      y_0^* ( c_2( H_1^2 + zm_2 _1^2 +(zm_2)'_1 )
         +( c_1 + c_2 m_1'/m_1)(  H_1 + zm_2 _1 ) 
      ) ϑ_0
         + ∑_t=1,2y_t^*   (  H_1 +zm_2 _1 )  ϑ_t)
      ^l-1.
  
  For the coefficient c_1 + c_2 m_1'/m_1 on the RHS of (<ref>), by elementary computations, we see 
  
      c_2+c_2m_1'(θ(d_i))/m_1(θ(d_i)) =  c_1,
  
  where c_1 is defined in def:wtc12 and c_1,c_2 are given in (<ref>).  If we substitute  z=θ(d_i)+ N^-K into m_1', m_1 above, since the imaginary part can be taken arbitrarily small, we can get c_1+c_2m_1'/m_1=  c_1 + O_≺(N^- K) for some sufficient large K. Thus in later estimates, we shall simply replace the coefficient c_1+c_2m_1'/m_1 in (<ref>) by c_1 and the resulting error will be negligible by taking K sufficiently large. Similarly, in the following, once we do the substitution to get the value of certain function of z, we simply take z=θ(d_i) for the computation, up to negligible error. 
  
  To  ease the notation, we use c_2=c_2 (see the definition in def:wtc12) in 2020050701. For t=0,1,2, we introduce the following functions in terms of _1  
   
      T_t(_1)=
       {[ ∑_a=1^2   c_a _1^a,             if t=0;;                 _1,           if t=1,2. ].
  
  With the above notations, we can simply write 2020050701 as 
  
      𝔼(^l)=     -  m_1/Δ(d_i)∑_t=1^3  y_t^* H T_t(_1)ϑ_t^l-1
         -   m_1/Δ(d_i)( y_0^* ( c_2 (zm_2_1^2+ (zm_2)'_1 ) +  c_1 zm_2_1 ) ϑ_0
             + ∑_t=1,2y_t^*zm_2_1 ϑ_t) ^l-1.
  
  
  
  Now, we apply the cumulant expansion  to the terms -  m_1/Δ(d_i)y_t^* H T_t(_1)ϑ_t for t=0,1,2. For simplicity, we use the following shorthand notation for the summation
  
      ∑_q,k=∑_q=1^M ∑_k=1^N,
  
  and similar shorthand notations are also used for single sum. 
  By Lemma <ref>,  we have 
  
      m_1/Δ(d_i)y_t^* H T_t(_1)ϑ_t^l-1 = 
       m_1/Δ(d_i)∑_q,k y_tq  x_qk (X^*T_t(_1) ϑ_t)_k^l-1
                 =  m_1/Δ(d_i)𝔼∑_q,ky_tq∑_α=1^3κ_α+1(x_qk)/α!∂^α/∂ x_qk^α((X^*T_t(_1) ϑ_t )_k ^l-1)+ℛ_t,
  
  where ℛ_t satisfies
  
      |ℛ_t|≤   |m_1/Δ(d_i)|∑_q,k(C𝔼(|x_qk|^5) 𝔼(sup_|x_qk|≤ c|y_tq∂^4/∂ x_qk^4( (X^*T_t(_1) ϑ_t )_k ^l-1) | ) 
         +C𝔼(|x_qk|^51(|x_qk|>c)) 𝔼(sup_x_qk∈ℝ|y_tq∂^4/∂ x_qk^4((X^*T_t(_1) ϑ_t )_k ^l-1) |) )
  
  for any  constant c>0 and  some  constant C>0.
  
  
  
  
  
  By the product rule, we have 
  
      ∂^α/∂ x_qk^α((X^*T_t(_1) ϑ_t )_k ^l-1)= ∑_cα_1,α_2≥ 0
      α_1+α_2=ααα_1∂^α_1(X^*T_t(_1) ϑ_t )_k /∂ x_qk^α_1∂^α_2^l-1/∂ x_qk^α_2.
  
  For t=0,1,2,  we set the notation
   
      h_t(α_1,α_2):=m_1/Δ(d_i)∑_q,ky_tqκ_α_1+α_2+1(x_qk)/(α_1+α_2)!α_1+α_2α_1∂^α_1(X^*T_t(_1) ϑ_t )_k /∂ x_qk^α_1∂^α_2^l-1/∂ x_qk^α_2.
  
  Note that h_t(α_1,α_2)  depends on   l and i. However, we drop this dependence for brevity.
  
  Using (<ref>)-(<ref>), we can now write
  
      m_1/Δ(d_i)y_t^* H T_t(_1)ϑ_t^l-1   =∑_cα_1,α_2≥ 0
      1≤α_1+α_2≤ 3𝔼h_t(α_1,α_2)+ℛ_t.
   
  
  In the sequel, we estimate h_t(α_1,α_2)  and the remainder terms ℛ_t for t=0, 1, 2.   We collect the estimates  in the following lemma,  whose proof will be postponed to the end of this section. Before we state the lemma, let us first recall the shorthand notation δ_i0=d_i-√(y).
    Let l in (<ref>) be any fixed positive integer. 
  With the convention m_a^(-1)/(-1)!=1 for a=1,2, we have the following estimates on h_t(α_1,α_2)  and ℛ_t where t=0,1,2. 
  
  
    (1): For h_t(α_1,α_2), the non-negligible terms are
  
      h_0(1,0)=-  m_1/Δ(d_i)(  c_1(y_0^*_1ϑ_0) (zm_2)+  c_2(y_0^*_1ϑ_0) (zm_2)' 
                     +  c_2(y_0^*_1^2ϑ_0) (zm_2)) ^l-1
      +O_≺(d_i^3/2δ_i0^-3/2 N^-1/2), 
         h_0(0,1)= - (l-1)m_1/Δ(d_i)^2(∑_a,b =1^2 c_a  c_b (∑_c a_1,a_2≥ 1,
       a_1+a_2=a+1
      m_1^(a_1-1) (zm_1)^(b+a_2-1)/(a_1-1)! (b+a_2-1)!)
                     ×((y_0^*y_0) (ϑ_0^*ϑ_0) +  (ϑ_0^*y_0)^2)  
               + ∑_s=1^2 m_1 ( ∑_b=1^2 c_b (zm_1)^(b)/b!)
       ( (y_0^*y_s) (ϑ_0^*ϑ_s)+(y_0^*ϑ_s)(ϑ_0^*y_s))
      )^l-2
               +O_≺(d_i^3/2δ_i0^-3/2 N^-1/2), 
         h_0(1,2)=- κ_4 (l-1) m_1/Δ(d_i)^2∑_s=1^2  c_s((zm_2m_1)^(s-1)/(s-1)!) 
      (∑_b=1^2 c_b (zm_2m_1^2)^(b-1)/(b-1)!𝐬_2,2(y_0,ϑ_0)
               +∑_s=1^2 (zm_2m_1^2)   𝐬_1,1,1,1(y_0,ϑ_0, y_s,ϑ_s)
       )^l-2  + O_≺ (N^-1/2),
  
   and for t=1,2, 
  
      h_t(1,0)   =-  m_1/Δ(d_i)y_t^* _1 ϑ_t (zm_2) ^l-1 + O_≺(d_i^3/2δ_i0^-3/2 N^-1/2), 
      
      h_t(0,1)    = - (l-1)/Δ(d_i)^2(∑_a=1^2 c_a ((zm_1)'m_1^2)^(a-1)/a!((y_t^*y_0)(ϑ_t^*ϑ_0) + (y_t^*ϑ_0)(ϑ_t^*y_0) )
              + ∑_s=1^2  m_1^2 (zm_1)'  ((y_t^*y_s) ( ϑ_t^*ϑ_s)+
      (y_t^*ϑ_s) (ϑ_t^*y_s))
      )^l-2
              +O_≺(d_i^3/2δ_i0^-3/2 N^-1/2),
      
      
      h_t(1,2)   =-κ_4(l-1)/Δ(d_i)^2 (zm_2m_1^2)
       (∑_b=1^2 c_b (zm_2m_1^2)^(b-1)/(b-1)!𝐬_1,1,1,1(y_t,ϑ_t, y_0,ϑ_0) 
              +∑_s=1^2 (zm_2m_1^2) 𝐬_1,1,1,1(y_t,ϑ_t, y_s,ϑ_s)
      )^l-2  + O_≺ (N^-1/2).
  
  
  
  
  
  
  
    (2):
  Except for the above terms,
   all the other  h_t(α_1, α_2) terms with α_1,α_2≥ 0 and α_1+α_2≤ 3 can be bounded by O_≺(N^-1/2).
  
  
  
  
    (3): For the remainder terms, we have 
  
      ℛ_t=O_≺(N^-1/2).
   
  
  
  
  
   
  Now we show the proof of Proposition <ref>,  based on Lemma <ref>.
  
  First we show the proof of (<ref>). 
  Using  Lemma <ref>  with l=1, we can rewrite (<ref>) as
  
      m_1/Δ(d_i)𝔼y_t^*HT_t(_1)ϑ_t=𝔼h_t(1,0)+O_≺(N^-1/2).
  
  Plugging  the above estimate into (<ref>) with l=1, we obtain 
  
      𝔼=-(𝔼h_0(1,0)+ m_1/Δ(d_i)𝔼(  c_1(y_0^*_1ϑ_0) zm_2 +  c_2(y_0^*_1ϑ_0) (zm_2)' +  c_2(y_0^*_1^2ϑ_0) zm_2) 
             +𝔼h_1(1,0)+zm_2m_1/Δ(d_i)𝔼y_1^*_1ϑ_1+ 𝔼h_2(1,0)+zm_2m_1/Δ(d_i)𝔼y_2^*_1ϑ_2) 
              +O_≺(N^-1/2 ).
  
  We substitute the estimates  (<ref>) and (<ref>)  with l=1 into the above estimate and immediately get
  
      𝔼=O_≺(N^-1/2δ_i0^-3/2d_i^3/2).
  
  This proves (<ref>). 
  
  
  Next we turn to prove (<ref>). By (<ref>) and  Lemma <ref>, 
   we observe that 
  
      m_1/Δ(d_i)𝔼y_t^*HT_t(_1) ϑ_t^l-1
      =𝔼 h_t(1,0) + 𝔼 h_t(0,1) + 𝔼 h_t(1,2) +O_≺(N^-1/2 )
  
  for t=0,1,2. Plugging 19072402 into (<ref>), together with the estimates (<ref>) and (<ref>), we get 
  
      𝔼(^l)=   -(𝔼h_0(0,1)+𝔼h_0(1,2)+𝔼h_1(0,1)+𝔼h_1(1,2)+ 𝔼 h_2(0,1) + 𝔼 h_2(1,2) ) 
          + O_≺(N^-1/2δ_i0^-3/2d_i^3/2).
  
  It remains to compute the explicit expression for the RHS of the above equation. First, using rankresth0(0,1) and rankrestht(0,1), we get
  
      -(𝔼h_0(0,1)+𝔼 h_1(0,1) + 𝔼 h_2(0,1) )
         =l-1/Δ(d_i)^2( (‖y_0 ‖^2 +(y_0^* _𝖨^0)^2) ∑_a,b =1^2 c_a  c_b  ∑_c a_1,a_2≥ 1,
       a_1+a_2=a+1
      m_1m_1^(a_1-1) (zm_1)^(b+a_2-1)/(a_1-1)! (b+a_2-1)!
             +
      (‖y_1 ‖^2 +( y_1^* ς_𝖨^0)^2)  (m_1^2(zm_1)')
      
            +( y_1^*y_2 (ς_𝖨^0)^* _𝖨^0+  y_1^*_𝖨 ^0 y_2^*ς_𝖨^0) 
      (2m_1^2(zm_1)')+(‖y_2‖^2 +( y_2^* w_𝖨^0)^2)  (m_1^2(zm_1)')
            +( y_0^*y_1(w_𝖨^0)^*ς_𝖨^0 +  y_0^*ς_𝖨^0 y_1^*w_I^0 + y_0^*y_2‖w_𝖨^0‖ ^2 +  y_0^*w_𝖨^0 y_2^*w_I^0)
             ×(∑_a=1^2 c_a ((zm_1)'m_1^2)^(a-1)/a! + ∑_a=1^2  c_a m_1^2(zm_1)^(a)/a!)
       ) 𝔼𝒫^l-2 + O_≺(d_i^3/2δ_i0^-3/2 N^-1/2).
  
  Recall the definitions for y_0, y_1 and y_2 in def:y012 and the matrix ℳ_𝖨 in (<ref>) and (<ref>).  By elementary calculation, we arrive at
  
      -(𝔼h_0(0,1)+𝔼 h_1(0,1)+ 𝔼 h_2(0,1))  = (l-1) 𝐜^* ℳ_𝖨𝐜𝔼𝒫^l-2+ O_≺(d_i^1/2δ_i0^-1/2 N^-1/2).
  
  Next, by (<ref>) and (<ref>) , we have
  
      -(𝔼h_0(1,2)+𝔼h_1(1,2)+ 𝔼h_2(1,2)) 
         =(l-1) κ_4/Δ(d_i)^2( 𝐬_2,2(y_0, _𝖨^0) ( ∑_s=1^2  c_s m_1(zm_2m_1)^(s-1)/(s-1)!) 
      (  ∑_s=1^2  c_s (zm_2m_1^2)^(s-1)/(s-1)!)
         +( 𝐬_1,1,1,1(y_0, 𝐰_I^0,y_1, ς_I^0 ) +   𝐬_1,1,2(y_0,y_2, 𝐰_I^0 ))
      zm_2m_1^2
             ×(  ∑_s=1^2  c_s m_1(zm_2m_1)^(s-1)/(s-1)!+  ∑_s=1^2  c_s (zm_2m_1^2)^(s-1)/(s-1)!)
           +  (𝐬_2,2(y_1, ς_I^0)+ 𝐬_2,2(y_2, 𝐰_I^0) )(zm_2m_1^2)^2 + 𝐬_1,1,1,1(y_1, ς_I^0,y_2, 𝐰_I^0 ) (2(zm_2m_1^2)^2)
      ) 𝔼𝒫^l-2,
  
  which, by the definitions of y_0, y_1 and y_2 in def:y012 and the matrix 𝒦_𝖨 in (<ref>)-(<ref>), can be simplified to
  
      -(𝔼h_0(1,2)+𝔼h_1(1,2)+ 𝔼h_2(1,2)) 
      =(l-1) κ_4 𝐜^* 𝒦_𝖨𝐜𝔼𝒫^l-2.
  
  Combining ViI and ViJ, we complete the proof of  (<ref>) in Proposition <ref>. Hence, we conclude the proof of  Proposition <ref>. 
  
  
  
  
  The rest of this section is devoted to the proof of Lemma <ref>. It is convenient to first introduce the next lemma, which will be used  to control the negligible terms in the proof of Lemma <ref>. 
  
  
  
  For a fixed integer n≥ 1, let η_i=(η_i1,…, η_iM)^*∈ℂ^M, i∈ 0, n be any given deterministic vectors with max_iη_i≤ C for some positive constant C.  For positive integers s_0,s_1,⋯,s_n∈{1,2} and a=0,1, t=0,1,2, we have the following estimates:
  
      ∑_q|η_0q(_1^s_0)_qq)^a(∏_k=1^n(_1^s_kη_k)_q)|=O_≺(d_i^-n-a(d_i-√(y))^-∑_k=1^n (s_k-1)- a(s_0-1)) ,
         |∑_k((X^*_1^s_0X)_kk)^a(X^*T_t(_1)η_1)_k|=O_≺((d_i-√(y))^-1/2-a(s_0-1)d_i^-1/2-a).
  
  
  
  The first estimate Importestsum1 can be proved by (<ref>) and the isotropic local law (<ref>) as follows: 
  
      ∑_q|η_0q((_1^s_0)_qq)^a(∏_k=1^n((_1^s_kη_k)_q)|
         =∑_q|η_0q((_1^s_0)_qq)^a(∏_k=1^n⟨𝐞_q, _1^(s_k-1)/(s_k-1)!η_k ⟩)|
         =∑_q|η_0q(m_1^(s_0-1)/(s_0-1)!+O_≺(Δ(d_i)/Υ^s_0-1√(N)))^a ∏_k=1^n(m_1^(s_k-1)/(s_k-1)!η_kq+O_≺(Δ(d_i)/Υ^s_k -1√(N)))|
         =O_≺(d_i^-n-a(d_i-√(y))^-∑_k=1^n (s_k-1)- a(s_0-1)),
  
  where in the last step we used the first and  last estimates in estm1m2* and the fact |z|≍ |θ(d_i)|≍ d_i.
  
  f
  Recall  the  definition of T_t(_1) from (<ref>). To prove Importestsum2, we first claim that for t=0,1,2,  
  
      |∑_k(X^*T_t(_1)η_1)_k|= O_≺((d_i-√(y))^-1/2d_i^-1/2).
  
  Notice that for t=1,2, by weak_est_XG,
  
      |∑_k(X^*T_t(_1)η_1)_k|=|∑_k(X^*_1η_1)_k|=1_N^*X^*_1η_1=O_≺((d_i-√(y))^-1/2d_i^-1/2),
  
  where we recalled the notation √(N)1_N∈ℝ^N for the all-1 vector. For t=0, by definition, T_0(_1)=  c_1_1 +  c_2 _1^2 where c_1,2 are defined in def:wtc12. If d_i≤ K with sufficiently large constant K>0, then similarly to the above estimate, we have 
  
      |∑_k(X^*T_0(_1)η_1)_k|    ≤ c_1 1_N^*X^*_1η_1 +  c_2 1_N^*X^*_1^2η_1
         = O_≺((d_i-√(y))^-1/2d_i^-1/2).
   
  However, in the case that d_i>K with sufficiently large constant K>0, c_1,2 are no longer bounded. Instead, we do the expansion for _1, _1^2 around -1/θ(d_i) and 1/(θ(d_i))^2, respectively,  to get a cancellation. More specifically, 
  
      |∑_k(X^*T_0(_1)η_1)_k|    =  1_N^*X^*(θ(d_i)_1+ θ(d_i)^2 _1^2)η_1 + O_≺(d_i^-1) 
         = 1_N^*X^*Hη_1 /θ(d_i)+ O_≺(d_i^-1)= O_≺(d_i^-1).
  
  In the last two steps above, we used 
  
      1_N^*X^*H^aη_1 =  O_≺(N^-1/2)   for any a∈ℤ^+,
    
  which can be checked easily by the moment method. 
  
  
  
  
  Hence, we conclude the proof of est:sumXTG.
  Further,  by relationXG and the isotropic local law Theorem <ref>, we get
  
      |∑_k((X^*_1^s_0X)_kk)^a(X^*T_t(_1)η_1)_k|=|∑_k((_2^s_0-1+z _2^s_0 )_kk)^a(X^*T_t(_1)η_1)_k|
         =|∑_k(m_2^(s_0-2)/(s_0-2)!+zm_2^(s_0-1)/(s_0-1)!+O_≺(N^-1/2 d_i^s_0δ_i0^-2(s_0-1)Δ(d_i) ))^a ((X^*T_t(_1)η_1)_k|
         =|∑_k((1+zm_2)^(s_0-1)/(s_0-1)!+O_≺(N^-1/2 d_i^s_0δ_i0^-2(s_0-1)Δ(d_i) ))^a ((X^*T_t(_1)η_1)_k|
         =O_≺((d_i-√(y))^-1/2-a(s_0-1)d_i^-1/2-a),   a=0,1,
  
  by using   the convention m_2^(-1)/(-1)!=1, the product rule and also noticing that 1+zm_2= O(1/d_i) and (zm_2)'= O(d_i^-1(d_i-√(y))^-1) 
    from estm1m2* and estm1m2**. This concludes the proof of Lemma <ref>.  
  
  
  With Lemma <ref>, we can now prove Lemma <ref>.
   In this proof, we fix l and i. First, we compute h_t(1,0) for t=0,1,2. Recall the definition in rankrdefhts and the expression of T_t(_1) in def:Tt(mG).  We need to compute separately for t=0 and t=1,2. By deriXG^l, we have
  
  
      h_0(1,0)    =  m_1/Δ(d_i) ∑_q,k y_0q∂(X^* (∑_s=1^2  c_s _1^s) ϑ_0 )_k/∂ x_qk ^l-1
          =   m_1/Δ(d_i) ∑_q,k y_0q( ∑_s=1^2  c_s ((_1^sϑ_0 )_q -  ∑_a=1^2 ∑_cs_1,s_2≥ 1
       s_1+s_2=s+1 ( X^*_1^s_1𝒫_a^qk_1^s_2ϑ_0)_k ) )^l-1,
  
  where 𝒫_a^qk, a=1,2 are defined in P012. Taking the sum over q,k, we get 
  
      h_0(1,0)
         =√(N) m_1/Δ(d_i)∑_s=1^2  c_s  y_0^*_1^s ϑ_0^l-1
         -  m_1/Δ(d_i)∑_s=1^2  c_s    ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(y_0^* _1^s_1 X X^* _1^s_2ϑ_0+ y_0^* _1^s_2ϑ_0 Tr(X^* _1^s_1 X))^l-1.
  
  By the identity in (<ref>) and the explicit expression for c_1,2 in def:wtc12, we get 
  
      m_1/Δ(d_i)∑_s=1^2  c_s   ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1y_t^* _1^s_1 X X^* _1^s_2ϑ_0
         = m_1/Δ(d_i)( c_1(y_0^*_1XX^*_1ϑ_0)+ 2  c_2 (y_0^*_1XX^*_1^2ϑ_0)) 
         = m_1/Δ(d_i)( d_i^2+2d_iy+y/d_i+y(y_0^* (_1+ z_1')ϑ_0)+ (d_i^2-y)^2/d_i^2(y_0^* (2_1'+z_1”)  ϑ_0 ))  
       ,
  
  which, by isotropic local law weak_est_DG, can be estimated by
  
      m_1/Δ(d_i)∑_s=1^2  c_s   ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1y_t^* _1^s_1 X X^* _1^s_2ϑ_0
         =  m_1/Δ(d_i)( d_i^2+2d_iy+y/d_i+y((zm_1)'  y_0^* ϑ_0+ O_≺(Δ(d_i)d_i^2/δ_i0^2 ) )
             + (d_i^2-y)^2/d_i^2((zm_1)”y_0^* ϑ_0 + O_≺(Δ(d_i)d_i^3/δ_i0^4 )  ))
      
         =m_1/Δ(d_i)( d_i^2+2d_iy+y/d_i+y(zm_1)' + (d_i^2-y)^2/d_i^2 (zm_1)”) y_0^* ϑ_0 + O_≺( d_i^2/δ_i0^2 N) 
         =m_1/Δ(d_i)(-1/d_i+y) y_0^* ϑ_0 + O_≺( d_i^2/δ_i0^2 N) = O_≺( d_i^-1/2δ_i0^1/2/)
          + O_≺( d_i^2/δ_i0^2 N)
          = O_≺(  √(d_i/δ_i0N ) ).
   
  Here we used the facts (zm_1)'=1/(d_i^2-y) and (zm_1)”=-2d_i^3/(d_i^2-y)^3 together with the asymptotic expression for Δ(d) in Delta:asymp. The last step is due to the assumption asd.
  
  Substituting (<ref>) into (<ref>) and using the fact =O_≺(1), we get 
  
      h_0(1,0)   =√(N) m_1/Δ(d_i)∑_s=1^2  c_s  y_0^*_1^s ϑ_0^l-1
           -  m_1/Δ(d_i)∑_s=1^2  c_s    ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(y_0^* _1^s_2ϑ_0 Tr(X^* _1^s_1 X))^l-1
           + O_≺(  √(d_i/δ_i0 N) )
      .
  
  Using the second identity in (<ref>), and Theorem <ref>, more specifically, weak_est_m12N,  we have 
  
      m_1/Δ(d_i)∑_s=1^2  c_s    ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(y_0^* _1^s_2ϑ_0 Tr(X^* _1^s_1 X))^l-1
         = m_1/Δ(d_i)(  c_1(y_0^*_1ϑ_0) Tr (X^*_1X)/N+  c_2(y_0^*_1ϑ_0) Tr (X^*_1^2X)/N +  c_2(y_0^*_1^2ϑ_0) Tr (X^*_1X)/N) ^l-1
         = m_1/Δ(d_i)(  c_1(y_0^*_1ϑ_0) (1+zm_2)+  c_2(y_0^*_1ϑ_0) (zm_2)' + c_2(y_0^*_1^2ϑ_0) (1+zm_2)) ^l-1
            + O_≺(|/Δ(d_i) m_1^2  c_1(Tr (z_2)/N-zm_2)|)+ O_≺( |/Δ(d_i)  m_1^2  c_2 ( Tr (_2+z_2^2)/N - (zm_2)')|)
            +  O_≺( |/Δ(d_i) m_1m_1'  c_2 ( Tr (z_2)/N - zm_2)| ).
  
  Notice that c_1=O(d_i) and c_2=O_≺(δ_i0^2). 
   To estimate the error bounds in (<ref>), again,  we need  to separate the discussion into two cases:  d_i≤ K or d_i>K for sufficiently large constant K>0. In the case d_i≤ K, we can use the isotropic law est_m12N directly together with Δ(d_i)≍δ_i0^-1/2 and m_1=O(1), m_1'= O(1/  δ_i0) to get an O_≺(δ_i0^-3/2 N^-1/2) bound for all the error terms in (<ref>). In case d_i>K, we use the improved isotropic local law est_m12N for large z to get 
   
      |Tr (z_2)/N-zm_2 |=O_≺(N^-1 d_i^-1),  |  Tr (_2+z_2^2)/N - (zm_2)'|= O_≺(N^-1 d_i^-2).
  
   The above two estimates with the facts m_1(z)= O(1/d_i) and m_1'(z)= O(d_i^-1δ_i0^-1) yield the O_≺(N^-1/2)  bound for  all the error  terms in (<ref>) in the case d_i≤ K. To unify the two cases, we use the O_≺(d_i^3/2δ_i0^-3/2 N^-1/2) bound to estimate all the error terms in (<ref>).
  
  
  Then, we obtain 
  
      m_1/Δ(d_i)∑_s=1^2  c_s    ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(y_0^* _1^s_2ϑ_0 Tr(X^* _1^s_1 X))^l-1
         = m_1/Δ(d_i)(  c_1(y_0^*_1ϑ_0) (1+zm_2)+  c_2(y_0^*_1ϑ_0) (zm_2)' +  c_2(y_0^*_1^2ϑ_0) (1+zm_2)) ^l-1
             +O_≺( d_i^3/2δ_i0^-3/2 N^-1/2 ),
  
  which, together with 19072015, leads to 
  
      h_0(1,0)=   -  m_1/Δ(d_i)(  c_1(y_0^*_1ϑ_0) (zm_2)+  c_2(y_0^*_1ϑ_0) (zm_2)' +  c_2(y_0^*_1^2ϑ_0) (zm_2)) ^l-1
           +O_≺( d_i^3/2δ_i0^-3/2 N^-1/2 ).
  
  
  
  For the case t=1,2, we have
  
      h_t(1,0)    =  m_1/Δ(d_i) ∑_q,k y_tq∂(X^* _1 ϑ_t )_k/∂ x_qk ^l-1
          =   m_1/Δ(d_i) ∑_q,k y_tq((_1ϑ_t )_q -  ∑_a=1^2 ( X^*_1  𝒫_a^qk_1 ϑ_t)_k ) ^l-1
         =  m_1/Δ(d_i)y_t^* _1 ϑ_t -  m_1/Δ(d_i)  (  y_t^* _1 XX^*_1 ϑ_t ) -   m_1/Δ(d_i)  y_t^* _1 ϑ_t 1/NTr (X^*_1X) 
         = -  m_1/Δ(d_i)y_t^* _1 ϑ_t (zm_2) + O_≺(d_i^3/2δ_i0^-3/2 N^-1/2)
  
  where we used the estimates
  
      m_1/Δ(d_i)  (  y_t^* _1 XX^*_1 ϑ_t )    = 
        m_1/Δ(d_i) (zm_1)' y_t^* ϑ_t + O_≺(d_i/Nδ_i0^2)  
         = O_≺( d_i^-1/2δ_i0^-1/2N^-1/2)
  
  and 
  
      m_1/Δ(d_i)  y_t^* _1 ϑ_t 1/NTr (X^*_1X) =
         m_1/Δ(d_i)  y_t^* _1 ϑ_t (1+zm_2) + O_≺(d_i^3/2δ_i0^-3/2 N^-1/2).
  
  Both follow directly from the isotropic local laws weak_est_DG and weak_est_m12N.
  
  
  
  
  
  Next, we turn to estimate h_t(0,1), which by the definition in rankrdefhts reads
  
      h_t(0,1)   =(l-1)m_1/Δ(d_i)∑_q,ky_tq(X^*T_t(_1) ϑ_t)_k∂/∂ x_qk ^l-2.
  
  
  Using the formula eq:Pd1 to rankrhts(1,0), we get
  
      h_t(0,1)
         =- (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ((y_t^*_1^a_1y_0) (ϑ_t^*T_t(_1)XX^*_1^a_2ϑ_0)
                         +(y_t^*_1 ^a_2ϑ_0) (ϑ_t^*T_t(_1)XX^*_1^a_1y_0) )
         + ∑_s=1^2 ((y_t^*_1y_s)( ϑ_t^*T_t(_1)XX^*_1ϑ_s)+
      (y_t^*_1ϑ_s)(ϑ_t^*T_t(_1)XX^*_1y_s))
      ]^l-2.
  
  By (<ref>), we have
  
      _1^s_1XX^*_1^s_2 = _1^s_1+s_2-1 + z _1^s_1+s_2,      a=1,2.
  
  Hence, for t=0, by  (<ref>) and the bound ||≺ 1, we have 
  
      h_0(0,1)    = - (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ( m_1^(a_1-1)/(a_1-1)! (y_0^*y_0) ( ∑_b=1^2 c_b (zm_1)^(b+a_2-1)/(b+a_2-1)!) (ϑ_0^*ϑ_0)
         +m_1^(a_2-1)/(a_2-1)!(y_0^*ϑ_0) (∑_b=1^2 c_b(zm_1)^(b+a_1-1)/(b+a_1-1)!) (ϑ_0^*y_0) )
         + ∑_s=1^2 ( m_1(y_0^*y_s) ( ∑_b=1^2 c_b (zm_1)^(b)/b!)( ϑ_0^*ϑ_s)
                 +
      m_1(y_0^*ϑ_s) ( ∑_b=1^2 c_b (zm_1)^(b)/b!)(ϑ_0^*y_s))
      ]^l-2 + ℛ,
  
  where the error term ℛ is dominated by the sum of the absolute values of the following two terms 
  
      ℛ_1:=- (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ((y_0^*Ξ^(a_1-1)y_0)( ∑_b=1^2 c_b (zm_1)^(b+a_2-1)/(b+a_2-1)!) (ϑ_0^*ϑ_0)
               +(y_0^*Ξ^(a_2-1)ϑ_0) (∑_b=1^2 c_b(zm_1)^(b+a_1-1)/(b+a_1-1)!) (ϑ_0^*y_0) )
              + ∑_s=1^2 ((y_0^*Ξy_s) ( ∑_b=1^2 c_b (zm_1)^(b)/b!)( ϑ_0^*ϑ_s)+
      (y_0^*Ξϑ_s)( ∑_b=1^2 c_b (zm_1)^(b)/b!)(ϑ_0^*y_s))
      ],
  
  and 
  
      ℛ_2: = - (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ( m_1^(a_1-1)/(a_1-1)! (y_0^*y_0) ( ∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b+a_2-1)/(b+a_2-1)!]ϑ_0)
         +m_1^(a_2-1)/(a_2-1)!(y_0^*ϑ_0) (∑_b=1^2 c_b ϑ_0^*[   (zΞ)^(b+a_1-1)/(b+a_1-1)!] y_0 ) )
         + ∑_s=1^2 ( m_1(y_0^*y_s) ( ∑_b=1^2 c_b  ϑ_0^*[  (zΞ)^(b)/b!] ϑ_s) +
      m_1(y_0^*ϑ_s) ( ∑_b=1^2 c_b ϑ_0^*[  (zΞ)^(b)/b!] y_s))
      ].
  
  In the sequel, we prove both ℛ_1 and ℛ_2 are negligible. First, we apply weak_est_DG and the facts which can be checked by elementary computations
  
      ∑_b=1^2  c_b (zm_1)^(b)/b! = y(1+d_i)/(d_i+y) (d_i^2-y),    ∑_b=1^2  c_b (zm_1)^(b+1)/(b+1)! = - d_i^2y/(d_i+y)(d_i^2-y)^2
  
  to get 
      ℛ_1= O_≺(d_i^1/2δ_i0^-1/2 N^-1/2).
  
  
  Next, we turn to ℛ_2. If d_i≤ K with sufficiently large constant K>0, direct applications of the isotropic local law weak_est_DG yield that  |ℛ_2|=O_≺(δ_i0^-3/2N^-1/2). Hence, we focus on the case that  d_i is large, i.e. d_i>K with sufficiently large constant K>0. 
   Recall the new form of eigenvector empirical spectral distribution(VESD) of H with respect to two fixed unit vectors , from def:VESDnew,
  Then for any given unit vectors ,, we can derive 
  
      ∑_b=1^2 c_b ^* [  (zΞ)^(b)/b!] 
           = ∫( c_1/x-z +  c_1z/(x-z)^2  +  c_2 /(x-z)^2 +  c_2z/(x-z)^3)  d(F_1N^,(x) - F^,_1(x)) 
           ≍∫_λ_–N^-2/3+ τ/2^λ_+ + N^-2/3+ τ/2(- c_1/(x-z)^2 -2  c_1z+2 c_2/(x-z)^3   - 3 c_2z/(x-z)^4)(F_1N^,(x) - F^,_1(x))    d  x.
  
  Further by est:Kol uv, i.e. sup_x| F_1N^,(x)- F_1^, (x)|= O_≺(N^-1/2) and the simple estimate 
  
      - c_1/(x-z)^2 -2  c_1z+2 c_2 /(x-z)^3   - 3 c_2z/(x-z)^4= O(d_i^-2)
  
  which follows form the substitution of c_1,  c_2 and z≍ d_i, we  finally have
  
      ∑_b=1^2 c_b ^* [  (zΞ)^(b)/b!]  
      =O_≺ (d_i^-2N^-1/2).
  
  
   Analogously, we have 
  
      ∑_b=1^2 c_b ^*[  (zΞ)^(b+1)/(b+1)!] 
      = O_≺ (d_i^-3N^-1/2).
  
  Plugging 2020051101 back into def:mathcalR2, we see 
  
      ℛ_2=   - (l-1)m_1/Δ(d_i)^2[ ( ∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]ϑ_0)(y_0^*y_0) (c_1/2m_1+ c_2m_1') ] 
         - (l-1)m_1/Δ(d_i)^2[ ( ∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]y_0)(y_0^*ϑ_0) (c_1/2m_1+ c_2m_1') ]
         - (l-1)m_1/Δ(d_i)^2[ m_1( c_1/2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]ϑ_0 + c_2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b+1)/(b+1)!]ϑ_0)(y_0^*y_0)  ] 
         - (l-1)m_1/Δ(d_i)^2[ m_1( c_1/2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]y_0 + c_2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b+1)/(b!+1)]ϑ_0)(y_0^*ϑ_0)]
         + O_≺(N^-1/2).
  
  By 2020051101 and the fact that c_1m_1/2+ c_2m_1'= O_≺(1/d_i), we see that the first two terms on the RHS of 2020051102 are of order O_≺(N^-1/2). Similarly to the estimates of 2020051101 and 2020051101*, we also get 
  
      m_1( c_1/2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]ϑ_0 + c_2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b+1)/(b+1)!]ϑ_0) 
         ≍
       m_1 (c_1/2∫_λ_–N^-2/3+ τ/2^λ_+ + N^-2/3+ τ/2(- c_1/(x-z)^2 -2  c_1z+ 2 c_2/(x-z)^3   - 3 c_2z/(x-z)^4)(F_1N^,(x) - F^,_1(x))    d  x 
            + c_2 ∫_λ_–N^-2/3+ τ/2^λ_+ + N^-2/3+ τ/2(-2 c_1/(x-z)^3 -3  c_1z+ 3 c_2 /(x-z)^4 - 4 c_2z/(x-z)^5)(F_1N^,(x) - F^,_1(x))    d  x ).
  
  For z≍ d_i>K, expanding (x-z)^-i, i=1,⋯,5 at x=0 and plugging in the values of c_1,2,  c_1,2 defined in def:c12 and  def:wtc12, after elementary computations, we see that 
  
      c_1/2(- c_1/(x-z)^2 -2  c_1z+ 2 c_2/(x-z)^3   - 3 c_2z/(x-z)^4) 
         + c_2 (-2 c_1/(x-z)^3 -3  c_1z+ 3 c_2/(x-z)^4  - 4 c_2z/(x-z)^5) = O(d_i^-2),
  
  which together with est:Kol uv and  m_1= O(d_i^-1) leads to 
  
      m_1( c_1/2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b)/b!]ϑ_0 + c_2∑_b=1^2 c_b  ϑ_0^*[(zΞ)^(b+1)/(b+1)!]ϑ_0)=  O_≺(d_i^-3N^-1/2).
   
  
  
  Hence, we see the third and fourth terms on the RHS of 2020051102 are crudely bounded by O_≺(N^-1/2). Now, we conclude that |ℛ_2|= O_≺ (N^-1/2) for large d_i and  |ℛ_2|=O_≺(δ_i0^-3/2N^-1/2) for bounded d_i. This, together with |ℛ_1|=O_≺(d_i^1/2δ_i0^-1/2 N^-1/2), finally leads to 
  
      h_0(0,1)= - (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ( m_1^(a_1-1)/(a_1-1)! (y_0^*y_0) ( ∑_b=1^2 c_b (zm_1)^(b+a_2-1)/(b+a_2-1)!) (ϑ_0^*ϑ_0)
                    +m_1^(a_2-1)/(a_2-1)!(y_0^*ϑ_0) (∑_b=1^2 c_b(zm_1)^(b+a_1-1)/(b+a_1-1)!) (ϑ_0^*y_0) )
                 + ∑_s=1^2 ( m_1(y_0^*y_s) ( ∑_b=1^2 c_b (zm_1)^(b)/b!)( ϑ_0^*ϑ_s)
                    +
      m_1(y_0^*ϑ_s) ( ∑_b=1^2 c_b (zm_1)^(b)/b!)(ϑ_0^*y_s))
      ]^l-2 +O_≺(d_i^3/2δ_i0^-3/2 N^-1/2).
  
  
  
  
  For t=1,2, using the same arguments as for h_0(0,1), we can derive 
  
      h_t(0,1)= - (l-1)m_1/Δ(d_i)^2[∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ( m_1^(a_1-1)/(a_1-1)! (y_t^*y_0) ( (zm_1)^(a_2)/(a_2)!) (ϑ_t^*ϑ_0)
                 +m_1^(a_2-1)/(a_2-1)!(y_t^*ϑ_0) ((zm_1)^(a_1)/(a_1)!) (ϑ_t^*y_0) )
                 + ∑_s=1^2 ( m_1(y_t^*y_s)  (zm_1)' ( ϑ_0^*ϑ_s)+
      m_1(y_t^*ϑ_s)(zm_1)'(ϑ_0^*y_s))
      ]^l-2
                 +O_≺(d_i^3/2δ_i0^-3/2 N^-1/2).
  
  
  
  
  Next, we show  rankresth0(1,2) and  rankrestht(1,2). First,  by the definition in (<ref>), we have 
   
      h_t(1,2)
          =κ_4/2N^3/2m_1/Δ(d_i)∑_q,ky_tq∂ (X^*T_t(_1)ϑ_t )_k/∂ x_qk
           ×((l-1) ∂^2 /∂ x_qk^2^l-2 + (l-1)(l-2)(∂/∂ x_qk)^2^l-3)
         =: κ_4(l-1)𝒥_1t +κ_4(l-1)(l-2) 𝒥_2t,
   
  In the following, we estimate 𝒥_1t and  𝒥_2t for t=0,1,2. We only state the details for the case t=0 as proofs for the cases t=1,2 are analogous and even simpler according  to the the definition of T_t(_1) in (<ref>).
    
  To estimate 𝒥_10, using
   (<ref>) and (<ref>),  
  we see
    
      𝒥_10   =
        1/2N^3/2m_1/Δ(d_i)∑_s=1^2  c_s ∑_q,ky_0q∂ (X^* _1^s ϑ_0  )_k/∂ x_qk∂^2 /∂ x_qk^2^l-2
         =1/Nm_1/Δ(d_i)^2∑_s=1^2  c_s ∑_q,ky_0q((_1^sϑ_0 )_q-∑_a=1^2∑_cs_1,s_2≥ 1
       s_1+s_2=s+1 (X^*_1^s_1𝒫_a^qk_1^s_2ϑ_0 )_k) 
           ×[∑_b=1^2 c_b (∑_a_1,a_2=1^2∑_cb_1,b_2,b_3≥ 1;
      ∑_i=1^3b_i=b+2 y_0^*(∏_j=1^2(_1^b_j𝒫_a_j^qk))_1^b_3ϑ_0-∑_cb_1,b_2≥ 1;
       b_1+b_2=b+1 y_0^*_1^b_1𝒫_0^qk_1^b_2ϑ_0)
            + ∑_t=1^2  (∑_a_1,a_2=1^2 y_t^*(∏_j=1^2(_1𝒫_a_j^qk))_1ϑ_t - 
       y_t^*_1𝒫_0^qk_1ϑ_t) ] ^l-2.
   
  Further, we claim that
  
      𝒥_10   =1/Nm_1/Δ(d_i)^2∑_s=1^2  c_s∑_q,ky_0q((_1^s ϑ_0  )_q-∑_cs_1,s_2≥ 1
       s_1+s_2=s+1 (X^*_1^s_1𝒫_2^qk_1^s_2ϑ_0  )_k) 
           ×[∑_b=1^2 c_b  y_0^*(∑_cb_1,b_2,b_3≥ 1
      b_1+b_2+b_3=b+2_1^b_1𝒫_1^qk_1^b_2𝒫_2^qk_1^b_3-∑_cb_1,b_2≥ 1
      b_1+b_2=b+1_1^b_1𝒫_0^qk_1^b_2)ϑ_0
            + ∑_t=1^2  y_t^*(_1𝒫_1^qk_1𝒫_2^qk_1-_1𝒫_0^qk_1)ϑ_t] ^l-2 + O_≺(N^-1/2).
  
  To see the reduction from (<ref>) to (<ref>), we first observe from Lemma <ref> that |m_1|∼ d_i^-1. Further notice that the terms absorbed into O_≺(N^-1/2) 
   always contain some quadratic forms of X^*_1^a for some a≥1 and at least one quadratic form of _1^b for some b≥ 1. Then the isotropic local law (<ref>) and Remark <ref> can be applied to show that the quadratic forms of  X^*_1^a and _1^b  are bounded by O_≺(N^-1/2δ_i0^-1/2d_i^-1/2 (δ_i0^-2d_i)^a-1) and O_≺(d_i^-1δ_i0^-(b-1)), respectively,  for a, b=1,2. For instance,  we have 
  
      N^-3/2m_1/Δ(d_i)∑_q,ky_0q c_s (∑_cs_1,s_2≥ 1
       s_1+s_2=s+1 (X^*_1^s_1𝒫_1_1^s_2ϑ_0)_k)  ∂^2 /∂ x_qk^2^l-2
         ≺ N^-3/2| m_1/Δ(d_i)∑_q,k∑_cs_1,s_2≥ 1
       s_1+s_2=s+1  y_0q c_s  (X^*_1^s_1)_kq (X^*_1^s_2ϑ_φ(t) )_k∂^2 /∂ x_qk^2| 
         ≺ N^-1|m_1|/Δ(d_i)∑_cs_1,s_2≥ 1
       s_1+s_2=s+1  c_s(∑_q| y_tq| )∑_k| (X^*_1^s_1)_kq| | (X^*_1^s_2ϑ_0)_k| √(d_i-√(y)/d_i)
         =O_≺(N^-1/2).
  
  In the first and second steps above, we used the  bound |𝒫| ≺ 1 and (<ref>),  
   respectively. In the last step, we used the isotropic local law (<ref>) and also the fact  ∑_q|y_tq|≺. 
   The other negligible terms can be estimated similarly. We omit the details. 
  
  Plugging the definitions in (<ref>) into (<ref>) yields
   
      𝒥_10   = m_1/NΔ(d_i)^2∑_s=1^2  c_s ∑_q,ky_0q((_1^sϑ_0)_q-∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1X)_kk(_1^s_2ϑ_0)_q)
           ×[∑_b=1^2 c_b (∑_cb_1,b_2,b_3≥ 1
      b_1+b_2+b_3=b+2( y_0^*_1^b_1)_q (X^*_1^b_2X)_kk(_1^b_3ϑ_0)_q -∑_cb_1,b_2≥ 1
      b_1+b_2=b+1(y_0^*_1^b_1)_q (_1^b_2ϑ_0)_q)
            + ∑_t=1^2  (( y_t^*_1)_q (X^*_1X)_kk(_1ϑ_t)_q-(y_t^*_1)_q (_1ϑ_t)_q)
       ] ^l-2 + O_≺(N^-1/2).
  
  Again, applying the identities in Lemma <ref>, the isotropic local laws (<ref>), and (<ref>), we see 
   
      𝒥_10   =m_1/Δ(d_i)^2∑_s=1^2  c_s(m_1^(s-1)/(s-1)!-∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(m_2^(s_1-2)/(s_1-2)!+zm_2^(s_1-1)/(s_1-1)!)m_1^(s_2-1)/(s_2-1)!) 
         ×(∑_b=1^2 c_b [∑_cb_1,b_2,b_3≥ 1
      b_1+b_2+b_3=b+2m_1^(b_1-1)/(b_1-1)!m_1^(b_3-1)/(b_3-1)!(m_2^(b_2-2)/(b_2-2)!+zm_2^(b_2-1)/(b_2-1)!)-(m_1^2)^(b-1)/(b-1)!]
      ∑_q y_0q^2 ϑ_0q^2 
         +∑_t=1^2 (m_1^2(1+zm_2)-m_1^2) ∑_q y_0q y_tqϑ_0qϑ_tq)^l-2 + O_≺ (N^-1/2),
  
   which by elementary calculation can be rewritten as
   
      𝒥_10   = - m_1/Δ(d_i)^2∑_s=1^2  c_s((zm_2m_1)^(s-1)/(s-1)!) 
      ×(∑_b=1^2 c_b (zm_2m_1^2)^(b-1)/(b-1)!∑_q y_0q^2 ϑ_0q^2
              +∑_t=1^2 (zm_2m_1^2) ∑_q y_0q y_tqϑ_0qϑ_tq)^l-2 + O_≺ (N^-1/2).
  
   Next, we show that
   
      𝒥_20 =m_1/2N^3/2Δ(d_i)∑_s=1^2  c_s ∑_q,k y_0q∂ (X^*_1^s ϑ_0)_k/∂ x_qk(∂/∂ x_qk)^2^l-3 = O_≺(N^-1/2).
  
   
   
  With (<ref>), we write 𝒥_20 as 
  
      𝒥_20   =m_1/2N^3/2Δ(d_i)∑_s=1^2  c_s 
      ∑_q,ky_0q((_1^sϑ_0)_q-∑_a=1^2∑_c s_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1𝒫_a^qk_1^s_2ϑ_0)_k)(∂/∂ x_qk)^2^l-3
         =m_1/2N^3/2Δ(d_i)∑_s=1^2  c_s ∑_q,ky_0q((_1^s ϑ_0)_q-∑_c s_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1X)_kk(_1^s_2ϑ_0)_q)(∂/∂ x_qk)^2^l-3
         +O_≺(N^-1/2),
  
  where in the last step we bounded the a=1 terms by  O_≺(N^-1/2) since 
  
      |m_1/2N^3/2Δ(d_i)∑_s=1^2  c_s∑_q,ky_0q∑_c s_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1)_kq(X^*_1^s_2ϑ_0)_kq(∂/∂ x_qk)^2^l-3|
         ≺ N^-1Δ(d_i)^-1|m_1| ∑_s=1^2∑_c s_1,s_2≥ 1
      s_1+s_2=s+1 | c_s| ∑_k|(X^*_1^s_1)_kq||(X^*_1^s_2ϑ_0)_kq|
         ≺ N^-1(d_i-√(y))^-1/2d_i^1/2≺  N^-1/2.
  
  Here we used the  O_≺(1) bound for both  and ∂/∂ x_qk ((<ref>)) for the first step, (<ref>) for the second step and asd, Delta:asymp for the last step.
  
  Further, we have
  
      m_1/2N^3/2Δ(d_i)∑_s=1^2  c_s ∑_q,ky_0q((_1^s ϑ_0)_q-∑_c s_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1X)_kk(_1^s_2ϑ_0)_q)(∂/∂ x_qk)^2^l-3
         = m_1/2N^3/2Δ(d_i)∑_q,ky_0qϑ_0q(  c_1 (-zm_2m_1) +  c_2 (-zm_2m_1)')(∂/∂ x_qk)^2^l-3
           + O_≺(N^-1/2)
  
  by using weak_est_DG. Recall def:wtc12 and the explicit expressions of c_1,2. Elementary computations indicate 
  
      c_1 (-zm_2m_1) +  c_2 (-zm_2m_1)' =-y(d_i^2+2d_i+y)/d_i^2 (d_i+y)
      =O(d_i^-1),
  
   by which we can further estimate the RHS of 2020051201 as
   
      |m_1|N^-1/2Δ(d_i)^-1‖y_0‖ ‖ϑ_0 ‖ d_i^-1|∂/∂ x_qk| ^2=O_≺(N^-1/2).
  
  Combining 17092052, 2020051201 and 2020051202, we proved 17092051. And this completes the estimate  of h_0(1,2).
   
   
   For the case t=1,2, by similar arguments, we will have 
   
      h_t(1,2)
         =κ_4/2N^3/2m_1/Δ(d_i)∑_q,ky_tq∂ (X^*_1ϑ_t )_k/∂ x_qk((l-1) ∂^2 /∂ x_qk^2^l-2 + (l-1)(l-2)(∂/∂ x_qk)^2^l-3)
         =- κ_4(l-1)/Δ(d_i)^2 zm_2m_1^2
       (∑_b=1^2 c_b (zm_2m_1^2)^(b-1)/(b-1)!∑_q y_tq y_0qϑ_tqϑ_0q
             +∑_s=1^2 zm_2m_1^2 ∑_q y_tq y_sqϑ_tqϑ_sq)^l-2  + O_≺ (N^-1/2).
   
   We omit the details here since it follows the same arguments as for h_0(1,2).
   
   
  
  
   
  
  In the sequel,  we prove (2) of Lemma <ref>, i.e.,   we show that except for (<ref>)-(<ref>), all the other terms h_t(α_1,α_2) with α_1+α_2≤ 3 can be bounded by O_≺(N^-1/2).  We start with the case when α_1+α_2=2, i.e. (α_1,α_2)=(2,0),(0,2),(1,1). 
  First,  by (<ref>), we have 
  
      h_0(2,0)   = κ_3 m_1/2NΔ(d_i)∑_s=1^2 c_s∑_q,ky_0q∂^2 (X^*_1^sϑ_0)_k/∂ x_qk^2^l-1
         =κ_3m_1 /NΔ(d_i)∑_s=1^2 c_s∑_q,ky_0q(∑_a_1,a_2=1^2∑_cs_1,s_2,s_3≥ 1
      s_1+s_2+s_3=s+2(X^*_1^s_1𝒫_a_1^qk_1^s_2𝒫_a_2^qk_1^s_3ϑ_0)_k
            -∑_a=1^2∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(_1^s_1𝒫_a^qk_1^s_2ϑ_0)_q-∑_cs_1,s_2≥ 1
      s_1+s_2=s+1(X^*_1^s_1𝒫_0^qk_1^s_2ϑ_0)_k)^l-1.
  
   Note that each term above contains at least one quadratic form of X^*_1^a as a factor,  for some a≥ 1. This fact eventually leads to the O_≺(N^-1/2d_i^- 1) bound for all the terms above, by the isotropic local law.  More specifically, plugging the definitions in (<ref>) into (<ref>) and taking the sums, we can see that  the RHS of (<ref>) is a linear combination of the terms of the following forms 
  
      N^-1 c_s m_1/Δ(d_i)∑_q,ky_0q(X^*_1^s_1)_kq(X^*_1^s_2)_kq(X^*_1^s_3ϑ_0)_k ^l-1, 
         N^-1 c_s m_1/Δ(d_i)∑_q,ky_0q((X^*_1^s_1X)_kk)^a(X^*_1^s_2)_kq(_1^s_3ϑ_0)_q ^l-1,
         N^-1 c_s m_1/Δ(d_i)∑_q,ky_0q((X^*_1^s_1X )_kk)^a(_1^s_2)_qq(X^*_1^s_3ϑ_0)_k^l-1,      a=0,1.
  
  where s=1,2 and s= s_1+s_2+s_3-2 for 17092071 and s=a(s_1-1)+s_2+s_3-1
   for 17092072 and 17092070. 
   
  It suffices to estimate (<ref>)-(<ref>). First, notice that |^l-1|=O_≺(1),  |m_1|=O_≺(1/d_i) and the quadratic forms of (X^*_1) and (X^*_1^2) can be bounded by O_≺(N^-1/2(d_i-√(y))^-1/2d_i^-1/2) and O_≺(N^-1/2(d_i-√(y))^-5/2d_i^1/2) respectively.   By the Cauchy Schwarz inequality, ∑_q|y_tq|=O_≺ (). With the expression of c_1,2 in def:wtc12,  one see that the  term in (<ref>) is bounded by O_≺(N^-1(d_i-√(y))^-1).  Second, combining the following bounds
  
      |^l-1|=O_≺(1),    |(X^*_1X)_kk|=O_≺(d_i^-1), 
          |(X^*_1^2X)_kk|= O_≺(d_i^-1(d_i-√(y))^-1),
         |(X^*_1)_kq|= O_≺(N^-1/2(d_i-√(y))^-1/2d_i^-1/2), 
          |(X^*_1^2)_kq|=O_≺(N^-1/2(d_i-√(y))^-5/2d_i^1/2),
  
  together with the estimate of Δ(d_i) in Delta:asymp, we have for a=1,
  
      (<ref>)   =O_≺(N^-1/2 c_s d_i^s_2-2 (d_i-√(y))^-(s_1-1)-2(s_2-1)∑_q|y_tq(_1^s_3ϑ_φ(t))_q|)
         =O_≺(N^-1/2 c_s d_i^s_2-3 (d_i-√(y))^-(s-1)-(s_2-1)) = O_≺(d_i^-1N^-1/2).
  
  In the last two steps above, we used Importestsum1, c_1= O(d_i) and c_2= O((d_i-√(y))^2). More specifically, if s=1, then s_2=1, c_s d_i^s_2-3 (d_i-√(y))^-(s-1)-(s_2-1)= O(d_i^-1); if s=2, we have c_s d_i^s_2-3 (d_i-√(y))^-(s-1)-(s_2-1)=O(d_i^-(3-s_2)  (d_i-√(y))^2-s_2)< O(d_i^-1).  The case of a=0 follows similarly as
  
      (<ref>)=O_≺(N^-1/2 c_s d_i^s_2-1 (d_i-√(y))^-2(s_2-1)∑_q|y_tq(_1^s_3ϑ_φ(t))_q|)
       = O_≺(N^-1/2).
  
  Third, by Cauchy-Schwarz inequality and (<ref>), we have
  
      |N^-1 c_s m_1/Δ(d_i)∑_q,ky_0q((X^*_1^s_1X )_kk)^a(_1^s_2)_qq(X^*_1^s_3ϑ_0)_k|
         ≤ N^-1 c_s|m_1|/Δ(d_i)𝐲_0(∑_q(_1^s_2)_qq^2)^1/2|∑_k((X^*_1^s_1X )_kk)^a(X^*_1^s_3ϑ_0)_k|
         ≺ N^-1/2 c_s d_i^-2 (d_i-√(y))^-(s_2-1)Δ(d_i)^-1|∑_k((X^*_1^s_1X )_kk)^a(X^*_1^s_3ϑ_0)_k| 
         ≺ N^-1/2.
  
  In the last step above,  we get the O_≺(d_i^-a-1/2+s_3-1δ_i0^-1/2-a(s_0-1)-2(s_3-1)) bound for the term |∑_k((X^*_1^s_1X )_kk)^a(X^*_1^s_3ϑ_0)_k|, similarly to  (<ref>). Hence, we conclude h_0(2,0)=O_≺(N^-1/2). Similarly, for the cases of t=1,2, the estimates of h_t(2,0) are reduced to those of the forms 17092071-17092070 only with the coefficients c_s removed. The computations turn out to be even easier; the details are omitted. Therefore, we also have h_t(2,0)=O_≺(N^-1/2) for t=1,2.
  
  
  Next, in the case of  (α_1,α_2)=(0,2), by the definition in (<ref>), we have 
  
      h_t(0,2)
          =(l-1)(l-2)κ_3 m_1/2NΔ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k(∂/∂ x_qk)^2 ^l-3
             +(l-1)κ_3 m_1 /2NΔ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k∂^2/∂ x_qk^2^l-2.
  
  We then use the formula  in eq:Pd1. After expanding the first term in the RHS of rankrhts(0,2), one notices that it can be written as a linear combination of the terms of the  forms (by ignoring prefactor m_1/Δ(d_i)^3)
  
      ∑_qy_tq(_1η_1)_q(_1ψ_1)_q∑_k(X^*T_t(_1)ϑ_t)_k(X^*_1η_2)_k(X^*_1ψ_2)_k^l-3,  
         ∑_q,ky_tq (X^*T_t(_1)ϑ_t)_k∑_a=1^2 c_a ∑_c a_1,a_2≥ 1
       a_1+a_2=a+1 
      (_1^a_1η_1)_q (X^*_1^a_2η_2)_k
                         ×∑_b=1^2 c_b ∑_c b_1,b_2≥ 1
       b_1+b_2=b+1 
      (_1^b_1ψ_1)_q (X^*_1^b_2ψ_2)_k,
        
         ∑_q,ky_tq (X^*T_t(_1)ϑ_t)_k(∑_a=1^2 c_a ∑_c a_1,a_2≥ 1
       a_1+a_2=a+1 
      (_1^a_1η_0)_q (X^*_1^a_2η_0)_k)
       (_1ψ_s)_q (X^*_1ψ_s)_k,  
                      s=1,2.
  
  where the vectors η_α,ψ_α (α=1,2) take ϑ_s or y_t for s,t=0,1,2.
  We claim that  the above forms  (<ref>)- 19072080** are bounded by O_≺(N^-1/2d_i^-7/2(d_i-√(y))^-3/2 ) for all choices of η_α,ψ_α listed above. To see this, we first notice that  the isotropic local law  (<ref>) implies
  
      ∑_k(X^*_1ϑ_t)_k(X^*_1η_2)_k(X^*_1ψ_2)_k=O_≺(N^-1/2 (d_i-√(y))^-3/2d_i^-3/2 ).
  
  Plugging in the expressions of c_1,2, we see that  if d_i≤ K with sufficiently large constant K>0,
  
      (X^*T_0(_1)ϑ_0)_k=∑_s=1^2  c_s (X^*_1^s ϑ_0)_k = O_≺(N^-1/2  (d_i-√(y))^-1/2 d_i^-1/2).
   
  If  d_i > K with sufficiently large constant K>0, expanding _1, _1^2 around -1/θ(d_i) and 1/(θ(d_i))^2, respectively, we obtain 
  
      (X^*T_0(_1)ϑ_0)_k   =∑_s=1^2  c_s (X^*_1^s ϑ_0)_k
         = θ(d_i)(X^*(_1+ θ(d_i)_1^2)ϑ_0)_k + O_≺ (N^-1/2  (d_i-√(y))^-1/2 d_i^-1/2) 
         = (X^*Hϑ_0)_k/θ(d_i) + O_≺(N^-1/2d_i^-1) = O_≺(N^-1/2d_i^-1),
  
  where in the last step, we used (X^*H^αϑ_0)_k = O_≺(N^-1/2) for α∈ℤ^+ which can be proved simply by moment method. 
  
  
  
  Combining the two cases together with the isotropic local law for (X^*_1ϑ_t)_k, we get the estimate 
  
      (X^*T_t(_1)ϑ_t)_k= O_≺(N^-1/2  (d_i-√(y))^-1/2 d_i^-1/2),    for  t=0,1,2.
  
  This implies 
  
      ∑_k(X^*T_0(_1)ϑ_0)_k(X^*_1η_2)_k(X^*_1ψ_2)_k=O_≺(N^-1/2 (d_i-√(y))^-3/2d_i^-3/2 ).
  
  Further, from Importestsum1, we have 
  
      ∑_qy_tq(_1η_1)_q(_1ψ_1)_q=O_≺(d_i^-2 ).
   
  Combining 19072081, 2020052001,  (<ref>), and the fact ||≺ 1, we easily get 
      19072080=O_≺(N^-1/2d_i^-7/2(d_i-√(y))^-3/2)   for t=0,1,2.
   
  
  For  19072080* and 19072080**, the coefficients c_1,2 contain powers of d_i thus we need to handle them more carefully in the case of large d_i.  Notice that if d_i=O(1) then c_1,2= O(1), hence similarly  to the estimates 19072081, 2020052001 and (<ref>), we can easily derive the crude bound O_≺(N^-1/2d_i^-7/2(d_i-√(y))^-3/2) for 19072080* and 19072080* by using the isotropic local law. In the sequel, we only state the details  for the case d_i>K with sufficiently large constant K>0. First by plugging in the expressions of c_1,2 and recalling 2020052002, we see that  
  
      ∑_a=1^2 c_a ∑_c a_1,a_2≥ 1
       a_1+a_2=a+1 
      (_1^a_1η_1)_q (X^*_1^a_2η_2)_k
         = (c_1/2  (X^*_1η_2)_k +  c_2(X^*_1^2η_2)_k) (_1η_1)_q
       +   (c_1/2   (_1η_1)_q+  c_2 (_1^2η_1)_q)   (X^*_1η_2)_k
         = O_≺(N^-1/2d_i^-1)m_1 η_1q  +   (c_1/2   m_1+  c_2m_1') η_1q  O_≺(N^-1/2d_i^-1) + O_≺(N^-1/2d_i^-2) 
         = O_≺(N^-1/2d_i^-2)  η_1q,
  
  where we use the fact that c_1/2   m_1+  c_2m_1'= O(d_i^-1). Then by 2020052003 and est:XT012, we obtain that 
  
      19072080* = O_≺ (N^-1/2 d_i^-5) (∑_q y_tqη_1q ψ_1q) = O_≺ (N^-1/2d_i^-5 ),
  
  which follows from Cauchy-Schwarz inequality for ∑_q y_tqη_1q ψ_1q. Analogously, we also have 19072080** = O_≺ (N^-1/2d_i^-5 ). Note that we get the bound O_≺ (N^-1/2d_i^-5 ) in the large d_i case, which can be combined together with the case d_i=O(1) to get the crude bound O_≺(N^-1/2d_i^-7/2(d_i-√(y))^-3/2) for 19072080* and 19072080**. Now, multiplying with the prefactor m_1/Δ(d_i)^3 and using m_1=O(d_i^-1), we finally get the first term on the RHS of rankrhts(0,2) is of order O_≺ (N^-1/2).
  
   Analogously,  using the formula in  eq:Pd2, it is easy to see that the second term in the RHS of rankrhts(0,2) is a linear combination of the terms of the following forms containing coefficients c_1,2
    
      N^-1/2m_1/Δ(d_i)^2∑_b=1^2 c_b ∑_c
      b_1,b_2≥ 1
       b_1+b_2=b+1
      (∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q)
                         ×(∑_k(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2∑_b=1^2 c_b ∑_c
      b_1,b_2,b_3≥ 1
       b_1+b_2+b_3=b+1
      (∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q)
                         ×(∑_k(X^*_1^b_3X)_kk(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2∑_b=1^2 c_b ∑_c
      b_1,b_2,b_3≥ 1
       b_1+b_2+b_3=b+1
      (∑_qy_tq(_1^b_1η_1)_q)
                         ×(∑_k(X^*_1^b_2)_kq(X^*_1^b_3η_2)_k(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2∑_b=1^2 c_b ∑_c
      b_1,b_2,b_3≥ 1
       b_1+b_2+b_3=b+1
      (∑_qy_tq(_1^b_1)_qq)
                         ×(∑_k(X^*_1^b_2η_1)_k(X^*_1^b_3η_2)_k(X^*T_t(_1)ϑ_t)_k)^l-2,
  
  for η_0,η_1, η_2=ϑ_1, ϑ_2,y_1, y_2, y_3, a=0,1 and b_1,b_2,b_3=1 or 2 as well as the following forms  irrelevant to c_1,2,
    
      N^-1/2m_1/Δ(d_i)^2(∑_qy_tq(_1η_1)_q(_1η_2)_q)(∑_k(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2(∑_qy_tq(_1η_1)_q(_1η_2)_q)(∑_k(X^*_1X)_kk(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2(∑_qy_tq(_1η_1)_q)(∑_k(X^*_1)_kq(X^*_1η_2)_k(X^*T_t(_1)ϑ_t)_k)^l-2,
         N^-1/2m_1/Δ(d_i)^2(∑_qy_tq(_1)_qq)(∑_k(X^*_1η_1)_k(X^*_1η_2)_k(X^*T_t(_1)ϑ_t)_k)^l-2,
  
  for η_0,η_1, η_2=ϑ_1, ϑ_2,y_1, y_2, y_3, a=0,1 and b_1,b_2,b_3=1 or 2.
  
  
   We claim that all of the above terms can be bounded crudely by O_≺(N^-1/2 (d_i-√(y))^-1/2 d_i^1/2). First, recall the bound |𝒫|≺ 1.  The O_≺(N^-1/2(d_i-√(y))^1/2d_i^-1/2) bounds for (<ref>) and (<ref>) follow directly from |m_1|∼ 1/d_i, Delta:asymp, Importestsum1 and Importestsum2. From (<ref>) and the isotropic local law (<ref>), together with est:XT012, we see that (<ref>) is  bounded by O_≺(N^-1d_i^-1/2 (d_i-√(y))^-1/2). The estimate of (<ref>) is similar to that of (<ref>) by using the Cauchy-Schwarz inequality and the isotropic local law weak_est_XG. We thus omit the details.
   
    In the sequel, we bound the terms  eq:h02-1-eq:h02-3. The estimates for eq:h02-11-eq:h02-3 are the same as those of eq:h02-11*-eq:h02-3* by using isotropic local laws, est:XT012 and  the bounds c_1=O(d_i), c_2= O((d_i-√(y))^2), m_1=O(d_i^-1). We observe that each summand for b=1,2 in eq:h02-11-eq:h02-3 is crudely of order O_≺(N^-1/2), hence we can conclude the O_≺(N^-1/2) bounds for eq:h02-11-eq:h02-3.  As for eq:h02-1, in the case that d_i≤ K,  using the same arguments as for  eq:h02-1*,  one gets the bound O_≺(N^-1/2). We only need to show the case that  d_i>K, and we will observe the same cancellation as 2020052003. More specifically, for eq:h02-1, notice that 
   
      ∑_b=1^2 c_b ∑_c
      b_1,b_2≥ 1
       b_1+b_2=b+1
         (∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q)
         = ∑_qy_tqη_1qη_2q(c_1m_1^2+ 2c_2 m_1m_1') +O_≺(N^-1/2 d_i^-2)
         = O_≺(d_i^-2).
  
   This together with Importestsum2 leads to the bound O_≺(N^-1/2) for eq:h02-1.
    Hence, the second term in the RHS of rankrhts(0,2) is also of order O_≺(N^-1/2). 
  This together with the same bound for the first term in the RHS of rankrhts(0,2) leads to 
      h_t(0,2)=O_≺(N^-1/2),  for t=0,1,2.
  
  
  
  Next, we turn to  h_t(1,1).  By the definition in (<ref>), we have 
  
      h_t(1,1)=κ_3m_1/NΔ(d_i)∑_q,ky_tq∂ (X^*T_t(_1)ϑ_t)_k/∂ x_qk∂/∂ x_qk^l-2.
  
  Using deriXG^l and eq:Pd1, for t=0, we can write
  
      h_0(1,1)
         =-κ_3m_1/Δ(d_i)^2∑_q,ky_0q( (T_0(_1)ϑ_0)_q-∑_s=1^2  c_s ∑_a=1^2 ∑_cs_1,s_2≥ 1
      s_1+s_2=s+1
      (X^*_1^s_1^qk_a_1^s_2ϑ_0 )_k )
      
             ×( ∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ((y_0^*_1^a_1)_q(X^*_1^a_2ϑ_0)_k+(X^*_1^a_1y_0)_k(_1 ^a_2ϑ_0)_q)
              + ∑_t=1^2 ((y_t^*_1)_q(X^*_1ϑ_t)_k+(X^*_1y_t)_k(_1ϑ_t)_q) )^l-1.
  
  Note that by denoting  
  
      T_0(_1) :=c_1/2_1 + c_2 _1^2,    T_1(_1) = _1,
  
  we can represent the first term of ∂/∂ x_qk as 
  
      ∑_a=1^2 c_a ∑_ca_1,a_2≥ 1;
       a_1+a_2=a+1 ((y_0^*_1^a_1)_q(X^*_1^a_2ϑ_0)_k+(X^*_1^a_1y_0)_k(_1 ^a_2ϑ_0)_q)
         =∑_c
      t_1,t_2≥ 0
       t_1+t_2=1 [
      (y_0^* T_t_1(_1))_q(X^* T_t_2(_1)ϑ_0)_k + 
      (ϑ_0^* T_t_1(_1))_q(X^* T_t_2(_1)y_0)_k  ].
  
  Hence, it is easy to see that the RHS of def:h0(1,1) is a linear combination of the terms of  the following forms
  
      N^-1/2m_1/Δ(d_i)^2(∑_qy_tq(T_0(_1)ϑ_0)_q( T_t_1(_1)η_1 )_q)(∑_k (X^* T_t_2(_1)η_2 )_k)^l-1,
         N^-1/2m_1  c_s/Δ(d_i)^2(∑_qy_tq(_1^s_1ϑ_0)_q( T_t_1(_1)η_1 )_q)(∑_k 
      (X^*_1^s_2X)_kk(X^* T_t_2(_1)η_2 )_k)^l-1,
         N^-1/2m_1  c_s/Δ(d_i)^2(∑_qy_tq( T_t_1(_1)η_1 )_q)(∑_k(X^*_1^s_1)_kq(X^*_1^s_2ϑ_0)_k(X^* T_t_2(_1)η_2 )_k)^l-1,
  
  where t_1,t_2=0,1, s_1, s_2=1,2 satisfying s= s_1+s_2- 1∈{1,2} and η_1,η_2= y_0, y_1, y_2, ϑ_0, ϑ_1, ϑ_2. Recall the definitions of c_1,2 and c_1,2 (c.f. (<ref>), (<ref>)). We can write 
  
      (X^* T_0(_1)η_2 )_k = (X^*T_0(_1)η_2 )_k + O(1)×(X^*_1η_2 )_k.
  
  Similar estimates like Importestsum2 and est:XT012 can be deduced from replacing T_t(_1) by T_t(_1). Besides, we also have 
  
      ( T_0(_1)η_1 )_q =   ( T_0(_1)η_1 )_q  + O(1)×(_1η_1 )_q
         = ( c_1m_1 + c_2 m_1')η_1q + ( ( c_1 Ξ+  c_2 Ξ' ) η)_q  +O_≺( d_i^-1) η_1q
         =  ( ( c_1 Ξ+  c_2 Ξ' ) η)_q  +O_≺( d_i^-1) η_1q
         = O_≺( d_i^-1) η_1q
  
  which can be verified by directly applying isotropic local law weak_est_DG for the case d_i≤ K or involving eigenvector empirical spectral distribution to do further analysis of ( ( c_1 Ξ+  c_2 Ξ' ) η)_q for the case d_i>K. The details for the case d_i>K are analogous to that of 2020051101 and hence we skip them. 
  Then, with the above estimates, similar to the estimates of (<ref>)-(<ref>), all the  terms in terms:h0(1,1) can be bounded  by O_≺(N^-1/2). For the case t=1,2, similarly, we still see the forms in terms:h0(1,1) with T_0(_1) replaced by _1 and the coefficients c_s removed. These forms can be bounded by O_≺(N^-1/2) using the isotropic local laws weak_est_DG and weak_est_XG. We omit the details here. Hence, we proved h_t(1,1)=O_≺(N^-1/2). 
  
  Next, we  consider the other cases for α_1+α_3=3 except for (<ref>) and (<ref>), i.e. (α_1,α_2)=(3,0),(2,1),(0,3).  We start with the formulas  in (<ref>) and  (<ref>). Notice that according to the definition in (<ref>), we have 𝒪_2={(0,1),(0,2),(1,0),(2,0)} in (<ref>) and  (<ref>). 
  
  With (<ref>), we  can now estimate   h_t(3,0). By the definition in rankrdefhts, 
  
      h_t(3,0)=κ_4 m_1/3!N^3/2Δ(d_i)∑_q,ky_tq∂^3 (X^*T_t(_1)ϑ_t)_k/∂ x_qk^3^l-1.
  
  For t=0, after plugging in (<ref>) and taking the sums, one can check that except for the following type of terms  
  
      N^-3/2m_1  c_s/Δ(d_i)(∑_qy_0q(_1^s_3)_qq(_1^s_4ϑ_0)_q)(∑_k((X^*_1^s_1X)_kk)^a((X^*_1^s_2X)_kk)^b)𝒫^l-1
                           a,b=0,1,
   
  all the other terms of (<ref>) contain at least  one quadratic form of X^*_1^a for some a=1,2 and one quadratic form of _1^b for some b=1,2. Here, s_1,s_2, s_3,s_4 =1,2 satisfy s=s_3+s_4-1+ a(s_1-1)+ b(s_2-1)∈{1,2}.  Actually, by the isotropic local law (<ref>), those terms with at least  one quadratic form of X^*_1^a and one quadratic form of _1^b can all be dominated by a crude bound O_≺(N^-1/2). For instance, 
  
      N^-3/2m_1 c_s/Δ(d_i)∑_q,k y_0q(_1^s_1𝒫_1^qk_1^s_2𝒫_1^qk_1^s_3ϑ_0)_q^l-1
         = N^-3/2m_1  c_s/Δ(d_i)∑_q,k y_0q (_1^s_1)_qq (X^*_1^s_2)_kq (X^*_1^s_3ϑ_0)_k ^l-1
         =O_≺(N^-1(d_i-√(y))^-1/2d_i^-1/2),
  
  where s= s_1+s_2+s_3-2 ∈{1,2} and s_1,s_2,s_3=1,2.
  The other terms with at least  one quadratic form of X^*_1^a  and one quadratic form of _1^b  can be estimated similarly. And we use a crude bound O_≺(N^-1/2) to estimate them. We skip the details for those terms. 
  Further, using Importestsum1, Remark <ref>, c_1=O(d_i) and c_2=O((d_i-√(y))^2), one can easily get the terms in (<ref>) are bounded by O_≺(N^-1/2). Analogously to the above statements, we can derive the crude bound  O_≺(N^-1/2) for the case t=1,2. As a consequence, we get  h_t(3,0)=O_≺ (N^-1/2).
  
  For h_t(2,1), t=0,1,2, by the definition in (<ref>), we have
  
      h_t(2,1)=(l-1)κ_4m_1/2N^3/2Δ(d_i)∑_q,ky_tq∂^2 (X^*T_t(_1)ϑ_t)_k/∂ x_qk^2∂/∂ x_qk^l-2.
  
  Using the formula in (<ref>) and further the O_≺(N^-1/2(d_i-√(y))^-1/2-2(a-1)d_i^-1/2+(a-1)) bound for  the quadratic forms of X^*_1^a for  a= 1, 2, the O_≺((d_i-√(y))^-(b-1)d_i^-1) bound for  the quadratic forms of _1^b for b= 1, 2, it is not difficult to see 
   
      c_s ∂^2 (X^*_1^sϑ_t)_k/∂ x_qk^2=O_≺ (N^-1/2(d_i-√(y))^-1/2d_i^-1/2).
  
  Similarly, we can also obtain 
   
      ∂^2 (X^*T_t(_1)ϑ_t)_k/∂ x_qk^2= O_≺ (N^-1/2(d_i-√(y))^-1/2d_i^-1/2)    for  t=0,1,2.
  
  Then, from the above bound, |𝒫|=O_≺(1), |∂/∂ x_qk|=O_≺(1) ((<ref>)) and m_1= O(d_i^-1), one can conclude that  for t=0,1,2, h_t(2,1)=O_≺(N^-1/2 ).
  
  For h_t(0,3), we write
  
      h_t(0,3)   =κ_4 m_1/3!N^3/2Δ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k∂^3 ^l-1/∂ x_qk^3
         =κ_4(l-1)(l-2)(l-3)ℒ_1 + κ_4(l-1)(l-2)ℒ_2 + κ_4(l-1)ℒ_3,
  
  where
  
      ℒ_1:= m_1 /3!N^3/2Δ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k(∂/∂ x_qk)^3^l-4,
         ℒ_2:= m_1 /2!N^3/2Δ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k∂/∂ x_qk∂^2 /∂ x_qk^2^l-3,
         ℒ_3:=  m_1 /3!N^3/2Δ(d_i)∑_q,ky_tq(X^*T_t(_1)ϑ_t)_k∂^3 /∂ x_qk^3^l-2.
  
  
  First, note that  ℒ_1=, by using the facts |∂/∂ x_ik|≺ 1 ((<ref>)), |m_1|≍ d_i^-1 and  |(X^*T_t(_1)_i)_k|≺ N^-1/2(d_i-√(y))^-1/2d_i^-1/2 ((<ref>)), together with the Cauchy-Schwarz inequality. 
  
  Second, as for ℒ_2, we use the formula of ∂^2 /∂ x_qk^2 in (<ref>). Observe that, by the isotropic local laws  (<ref>) and (<ref>),  the terms in (<ref>) can be  bounded  by either O_≺(N^-1/2(d_i-√(y))^-1/2 d_i^1/2) or O_≺( (d_i-√(y))^1/2d_i^1/2).
   More precisely, the O_≺(N^-1/2(d_i-√(y))^-1/2 d_i^1/2) bound  is derived for the following forms 
  
      c_b/Δ(d_i) (X^*_1^b_1η_1)_k(X^*_1^b_2)_kq(_1^b_3η_2)_q,      c_b/Δ(d_i) (X^*_1^b_1η_1)_k(_1^b_2)_qq(X^*_1^b_3η_2)_k,
   
  and  the O_≺( (d_i-√(y))^1/2d_i^1/2) bound is obtained for the form 
      c_b Δ(d_i)^-1(_1^b_1η_1)_q(X^*_1^b_2X)_kk^a(_1^b_3η_2)_q.
   Here, η_1,η_2=y_1,y_2, y_3, _𝖨^0 or ς_𝖨^0,  a=0,1 and b_1,b_2,b_3=1 or 2 with b_1+b_2+b_3-2=b∈{1,2}.
   The contribution from the O_≺(N^-1/2(d_i-√(y))^-1/2 d_i^1/2) terms can be discussed similarly to ℒ_1. We thus omit the details. For the contribution from the  those  O_≺( (d_i-√(y))^1/2d_i^1/2) terms,   recalling the notations in def:wtTG, we notice that it suffices to consider the bound of the following forms 
  
      m_1 c_b/Δ(d_i)^3(∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q( T_t_1(_1)η_3)_q)
                     ×(∑_k(X^*T_t(_1) ϑ_t )_k(X^* T_t_2(_1)η_4)_k (X^*_1^b_3X)_kk) ^l-3,
         m_1 /Δ(d_i)^3∑_b=1^2 c_b
       ∑_c b_1,b_2≥ 1
       b_1+b_2=b+1(∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q( T_t_1(_1)η_3)_q)
                     ×(∑_k(X^*T_t(_1) ϑ_t )_k(X^* T_t_2(_1)η_4)_k ) ^l-3,
         m_1 /Δ(d_i)^3(∑_qy_tq(_1η_1)_q(_1η_2)_q( T_t_1(_1)η_3)_q)
                     ×(∑_k(X^*T_t(_1) ϑ_t )_k(X^* T_t_2(_1)η_4)_k((X^*_1X)_kk)^a) ^l-3,
  
  for a=0,1, t_1,t_2=0,1 and b=b_1+b_2+b_3-2∈{1,2}.
  By 2020052401, we get similar estimates to Importestsum1 for the sum ∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q( T_t_1(_1)η_3)_q.
  This bound, together with 2020052402, est:XT012, Remark <ref> and the estimates c_1=O(d_i), c_2=O((d_i-√(y))^-2), leads to the crude bound O_≺(N^-1/2) for the first and last forms in (<ref>). For the second form, one can refer to the estimate 2020052403. Analogously, we can also get 
  
      ∑_b=1^2 c_b
       ∑_c b_1,b_2≥ 1
       b_1+b_2=b+1(∑_qy_tq(_1^b_1η_1)_q(_1^b_2η_2)_q( T_t_1(_1)η_3)_q) = O_≺(d_i^-3),
  
  by which together with 2020052402, est:XT012, we have that the second form of (<ref>)  is bounded crudely by O_≺(N^-1/2).
   Hence, ℒ_2 is also bounded by O_≺ (N^-1/2).
  
  Finally, for ℒ_3, we use the bound in (<ref>). 
  Then following the same argument as for ℒ_1, one can prove that ℒ_3= O_≺(N^-1/2). 
  Thus, we have h_t(0,3)=O_≺ (N^-1/2 ) for t=0,1,2 and the proof of (2) in Lemma <ref> is complete. 
  
  
  Before we proceed to the proof of the remainder term ℛ_t, let us comment that, using the same reasoning as we  previously did for h_t(α_1,α_2) with α_1+α_2≤ 3, t=0,1,2, one can also get
  
      N^-2|m_1|/Δ(d_i)∑_q,k|y_tq∂^4 ((X^*T_t(_1)ϑ_t)_k^l-1)/∂ x_qk^4| = O_≺(N^-1/2),
  
  which implies that h_t(α_1,α_2) = O_≺(N^-1/2) for α_1+α_2 =4. The main tools are still Lemma <ref> and the isotropic local laws (<ref>)  and (<ref>). We omit the details. The necessary formulas for the fourth derivatives of (X^*_1^s)_kj and  are recorded in the Appendix <ref> for the readers' convenience. 
  
  
  
  
  
  In the end, we prove  (3) of Lemma <ref>. By Lemma <ref>, we can bound ℛ_t by
  
      |ℛ_t|≤|m_1|/Δ(d_i)∑_q,k𝔼(     N^-5/2sup_|x_qk|≤ N^-1/2+ϵ|  y_tq∂^4 ((X^*T_t(_1)ϑ_t)_k^l-1)/∂ x_qk^4| 
         +N^- Ksup_x_qk∈ℝ| y_tq∂^4 ((X^*T_t(_1)ϑ_t)_k^l-1)/∂ x_qk^4|),
  
  for any sufficiently large constant K. We evaluate the RHS of boundR_t term by term. 
  
  First, we claim that similarly to  (<ref>) we have 
  
      |m_1|/Δ(d_i)𝔼(N^-5/2sup_|x_qk|≤ N^-1/2+ϵ∑_q,k| y_tq∂^4 ((X^*T_t(_1)ϑ_φ(t))_k^l-1)/∂ x_qk^4|)
         =O_≺(N^-1/2).
  
  The difference between (<ref>) and (<ref>) is that we actually consider a random matrix X with the (q,k)-th entry deterministic while all the other entries random.  Using a regular perturbation argument through the resolvent expansion,   one can show that replacing one random entry x_qk in X by any deterministic  number bounded by N^-1/2+ϵ while keeping all the other X entries random will not change the isotropic local law. Thus the isotropic local law together with the deterministic bounds trivialbd_G1-trivialbd_XG1X leads to (<ref>).  
  
  
  For the second term on the RHS of boundR_t, we simply apply the crude deterministic bounds trivialbd_G1-trivialbd_XG1X. By choosing K sufficiently large, we can conclude that the second term in   boundR_t is negligible. 
    
  Thus |ℛ_t| = O_≺(N^-1/2). This completes the proof of  Lemma <ref>.
  
  
  
   
   
  
  § PROOFS OF THEOREMS IN SECTION <REF>
   
  
  In this section, we prove the two theorems in Section <ref>, Theorems <ref> and <ref>, and also their corollaries, Corollaries <ref> and <ref>. We also provide  a result equivalent to the joint eigenvalue-eigenvector distribution in the multiple case, which will be needed in the proofs of some of the aforementioned theorems and corollaries.
  In addition, the proofs of Theorems <ref> and <ref>  rely on Propositions <ref>-<ref>, among which the proofs of Propositions <ref>, <ref> and <ref> are analogous to that of Theorem <ref>, and the proof of Proposition <ref> is an extension of that of Lemma <ref>. 
  More specifically, Proposition <ref> is an extension of Theorem <ref>  by considering  more simple spikes and it will support the proof of the all-simple spikes case for both Theorem <ref> and Theorem  <ref>. Second, Proposition <ref> generalizes Theorem <ref> to allow multiple spikes. Consequently,  
  the case of all-equal spikes for both Theorem <ref> and Theorem <ref> will rely on Proposition <ref>.  It will be seen that the proof of Theorems <ref> and <ref> will also rely on Proposition <ref> which establish the expansion of the spiked eigenvalue estimator for a multiple spike. 
  
  
  
   
  Recall the notation 1_E as the indicator function of some event E, and also the notation ℐ^c= 1,r∖ℐ for any ℐ⊂1,r.
  Further, we recall the notation 𝖨≡𝖨(i) as the set of indices  of the multiple d_t's including d_i.  We remark here that throughout this section, ℐ is used to denote a generic subset of 1,r, which may include the indices of distinct d_t's. In contrast, the notation 𝖨≡𝖨(i) is always used to denote a set of indices of a multiple d_i. 
  
  
   Further, we raise the following notation as a rewriting of  def:v_I_varsigma_I in certain cases: For a fixed i∈1, r, and  any unit vector u∈Span{𝐯_j }_j∈ 1,M∖𝖨(i), we set 
   
      ς_i^u:=∑_j∈ 1,M∖{i}d_i√(d_j +1)/d_i -d_j⟨u, _j ⟩_j.
  
    Here we made the convention that ⟨u, _j ⟩/(d_i-d_j)=0, in case ⟨u, _j ⟩=d_i-d_j=0. Note that ς_i^u= ς_𝖨(i) (with w=u) (c.f. def:v_I_varsigma_I). Here we emphasize the dependence of the notation on w=u, and also 
   the dependence of 𝖨(i) on i. Such a rewriting will be mostly suitable for the discussion in this section. 
   
  
  Recall the projection P_ℐ defined in (<ref>).  In the sequel, we state and briefly prove the aforementioned propositions. We will consider the projections of vectors onto P_ℐ, and we are primarily interested in the case that either {d_t}_t∈ℐ are all distinct and well-separated or are all equal. We call the former as all-simple case, and the latter as all-equal case. Nevertheless, many of our results hold for more general ℐ in the sense that it can contain indices of both simple and multiple d_t's. 
   
    
   
  First, we consider an extension of Theorem <ref>. Suppose we are interested in the all-simple case, i.e., all the d_i's for i∈ℐ are simple, we have the following proposition.
  
  
  
  Suppose that Assumptions <ref> ,  <ref>,  and the setting (<ref>) hold.  In case d_i≡ d_i(N)→∞ as N→∞ for some i∈ℐ, we additionally assume that |y-1|≥τ_0 for some small but fixed τ_0>0. 
  We further assume that all d_i, i∈ℐ are simple. 
  Then for each i∈ℐ, we have the expansion of i-th eigenvalue μ_i  in expansion.eigv.
  Moreover, we have the following statements:
  
  (1). For any  _i with i∈ℐ,  we have
  
      ⟨_i, P_ℐ_i⟩=
         d_i^2-y/d_i(d_i+y)+ 1/√(N(d_i^2-y)) Θ__i^_i
         -1/N∑_t∈{i}^cd_id_t/(d_i-d_t)^2 (Π__t^_i)^2 
      +‖ς_j^v_i‖^2/N∑_j∈ℐ∖{i}1/d_j (Δ__j^v_i)^2  
          
      +O_≺(N^-(1+ε)∑_j∈ℐ∖{i}1/d_j‖ς_j^v_i‖^2) 
         + O_≺(N^-ε(N^-1/21/√(d_i^2-y) + 1/N∑_t∈{ i}^cd_id_t/(d_i-d_t)^2)).
  
   Further, {Φ_i}_i∈ℐ,{Θ__i^_i}_i∈ℐ, {Π__t^_i}_i∈ℐ , t∈{ i}^c, {Δ__j^v_i}_i,j∈ℐ, j≠ i are asymptotically jointly Gaussian distributed. Especially, 
  
      ( {Φ_i}_i∈ℐ, {Θ__i^_i}_i∈ℐ) ≃𝒩(0, C_ℐ)
  
  where C_ℐ is defined entrywise by  the RHS of the following equations
  
      cov (Φ_i, Φ_j) ≐1_{i=j}· 2(1+d_i^-1)^2 
         + κ_4 (1-yd_i^-2)^1/2 (1+d_i^-1)  (1-yd_j^-2)^1/2  (1+d_j^-1) s_2,2 (_i,_j),
         cov(Θ__i^_i, Θ__j^_j) ≐1_{i=j}·  2y 𝚑(d_i)^2 (1+ y 𝚑(d_i)^2)
         +  κ_4 (1-yd_i^-2)^1/2𝚏(d_i) (1-yd_j^-2)^1/2𝚏(d_j)  s_2,2 (_i,_j), 
         cov(Φ_i, Θ__j^_j) ≐1_{i=j}·  2y 𝚑(d_i)^2 (1+d_i^-1)  
         + κ_4 (1-yd_i^-2)^1/2 (1+d_i^-1)  (1-yd_j^-2)^1/2𝚏(d_j)s_2,2 (_i,_j),
  
  for i,j∈ℐ.  Here Φ_i is given in expansion.eigv. 
  
  (2). Let 𝒥 be any fixed index set. Further, let {u_j}_j∈𝒥⊂Span{𝐯_i }_i∈ 1,M∖ℐ be any  set of orthonormal vectors.  Then 
   
      ⟨u_j, P_ℐu_j⟩ =∑_i∈ℐ‖ς_i^u_j‖^2/Nd_i (Δ__i^u_j)^2  
      +O_≺(N^-(1+ε)∑_i∈ℐd_i^-1‖ς_i^u_j‖^2),  for j∈𝒥
  
  and ({Δ__i^u_j}_(j, i)∈𝒥×ℐ)≃𝒩(0,U_0). The  covariance  matrix U_0 is defined by  the RHS of the following equation
  
      cov(Δ__i_1^u_j_1, Δ__i_2^u_j_2)
      ≐1_{i_1=i_2}𝚑(d_i_1) ⟨(ς_i_1^u_j_1)^0,  (ς_i_2^u_j_2)^0 ⟩
             + κ_4√((d_i_1^2-y)𝚑(d_i_1))/d_i_1√((d_i_2^2-y)𝚑(d_i_2))/d_i_2 s_1,1,1,1(_i_1, _i_2, (ς_i_1^u_j_1)^0, (ς_i_2^u_j_2)^0 ),
  
  for j_1,2∈𝒥, i_1,2∈ℐ. Here we use the notation ( ς_i^u_j)^0 as the normalized ς_i^u_j similarly to def.nor.Varsigma. And  we  define  U to be the matrix obtained by multiplying the entry of U_0, the RHS of 200826010,  by ‖ς_i_1^u_j_1‖‖ς_i_2^u_j_2‖/√(d_i_1d_i_2).
   
  
    We remark here that  (<ref>) and the whole statement (2) in Proposition <ref>
  hold for general ℐ⊂ 1, r_0, with the same expression of the covariance structure C_ℐ and U_0. Here general ℐ means that it can contain the indices of either simple d_t's  or multiple ones,  or even both. The proof of such an extended version is nearly the same as the simple one. 
  
  
  
  
    The proof is basically the same as the one 
  of Theorem <ref>. The only change is that now one needs to put all Green function quadratic forms corresponding to distinct simple spikes  together and consider their joint Gaussianity. It will only require one to change the quadratic forms in the  definition of  𝒫 in (<ref>) and go through the recursive moment estimate again. Hence, we omit the details and conclude the proof. 
  
  
  Next, we consider the all-equal case, i.e.,   all d_i, i∈ℐ are identical. Analogously to the all-simple case, we can also estimate the multiple spike from the data. In the sequel, we present the detailed estimation of the unknown d_i in the multiple case. Recall the index set 𝖨(i) as the collection of indices for a multiple d_i. In the sequel, we are primarily interested in the case |𝖨(i)|>1, but the results apply to the degenerated (simple) case as well. For any t∈𝖨(i), according to Lemma <ref>, we have   |μ_t - θ(d_i)|≺ d_i^1/2δ_i0^1/2 N^-1/2. 
  We shall derive the expansion  for {μ_t}_t∈𝖨(i) and further  get the expression of ∑_t∈𝖨(i)μ_t /|𝖨(i)| which can be applied to estimate the unknown θ(d_i). Define 
  
      D_𝖨(i): = d_i/1+d_i I_|𝖨(i)| and   V_𝖨(i):= ({_t}_t∈𝖨(i)),
  
  where I_|𝖨(i)| means an identity matrix of dimension |𝖨(i)| and V_𝖨(i)  is the submatrix of V obtained by selecting the columns with  indices in 𝖨(i).  For convenience,  for the columns in V_𝖨(i), we keep their original  indices in V. With some abuse of notation, in this section, we will use the notation (λ_t (V_𝖨(i)^* A V_𝖨(i))) _t∈𝖨(i) to represent the family of eigenvalues of  V_𝖨(i)^* A V_𝖨(i), with  the  increasing order λ_t_1 (V_𝖨(i)^* A V_𝖨(i))≤λ_t_2 (V_𝖨(i)^* A V_𝖨(i)) if  t_1, t_2∈𝖨(i), t_1<t_2. We recall that δ_i0:= d_i-√(y), the distance of the multiple d_i to critical value √(y). With the above notations, we have the following proposition, whose proof will be stated in the end.
  
  Suppose that the assumptions in Theorem <ref> hold. Let i∈ 1, r_0. For any t∈𝖨(i),   we have
  
      μ_t = θ(d_i)-  (d_i^2-y)θ(d_i) λ_t ( V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i))   + O_≺(d_i^1/2δ_i0^1/2 N^-1/2 - ε)
  
  for some small fixed ε>0.
  
  
  
  
  
  Recall the representation in Lemma  <ref>. Similarly to Theorem <ref>, we can also derive the joint distribution of generalized components of eigenvectors and all  entries of the matrix  V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i) under appropriate scaling. The result is collected in the following proposition.  Set the |𝖨(i)|× |𝖨(i)| matrix
  
      Φ≡ (Φ_st)_s,t:= -√(N (d_i^2-y)) θ(d_i) V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i).
  
  We remark here that with certain abuse of notation Φ_ii≡Φ_i in (<ref>) in the simple case |𝖨(i)|=1. 
   
  Under the assumptions in Theorem  <ref>,  besides the expansions for  eigenvalues in Proposition <ref> and the expansion for generalized components weak_gf_decomp, for all of the upper triangular entries of Φ, i.e., {Φ_st}_s,t∈𝖨(i), s≤ t,  together with  the random terms in  weak_gf_decomp, we have 
  
      ( {{Φ_st}_t∈𝖨(i) , t≥ s}_s∈𝖨(i), Θ__𝖨(i)^, Λ_ς_𝖨(i)^, {Δ__t^}_t∈𝖨(i), {Π__j^}_j∈ (𝖨(i)) ^c) ≃𝒩(0, C_𝖨(i) ^)
  
  where C_𝖨(i) ^ is of size r+2+|𝖨(i) |^2 and the lower right (r+2)× (r+2) corner of C_𝖨(i) ^ is given by A_𝖨(i)^ + κ_4  d_i^2-y/d_i^2 B_𝖨(i)^ with A_𝖨(i)^ and B_𝖨(i)^ defined in (<ref>) and (<ref>) respectively. And the other entires of C_𝖨(i) ^ are defined by the RHS of the following equations
  
      cov (Φ_ℓ_1k_1, Φ_ℓ_2k_2 )≐1_{(ℓ_1, k_1)=(ℓ_2,k_2)} (1+1_ℓ_1=k_1 )(1+d_i^-1)^2 
                        + κ_4 (1-yd_i^-2)(1+d_i^-1)^2s_1,1,1,1 (_ℓ_1, _k_1, _ℓ_2, _k_2),
      
         cov (Φ_ℓ k, Θ__𝖨^ ) ≐ 2y𝚑(d_i)^2(1+d_i^-1)⟨_𝖨(i), _ℓ⟩⟨_𝖨(i), _k⟩
                       +κ_4 (1-yd_i^-2)(1+d_i^-1)𝚏(d_i) s_1,1,2(_ℓ,_k, _𝖨),
      
         cov (Φ_ℓ k, Λ_ς_𝖨^ ) ≐κ_4 (1-yd_i^-2)(1+d_i^-1)𝚐(d_i) s_1,1,1,1 (_ℓ,_k, ς_𝖨(i)^0, _𝖨),
      
         cov (Φ_ℓ k, Δ__t^ ) ≐κ_4 (1-yd_i^-2)(1+d_i^-1)√(𝚑(d_i) )s_1,1,1,1 (_ℓ,_k, _t, ς_𝖨(i)^0),  for t∈𝖨(i),
       
         cov (Φ_ℓ k , Π__j^ ) ≐κ_4 (1-yd_i^-2)(1+d_i^-1)𝚕(d_i) s_1,1,1,1 (_ℓ,_k, _j, _𝖨(i)),  for j∈𝖨(i) ^c,
  
  for ℓ, ℓ_1, ℓ_2, k, k_1, k_2∈𝖨(i) satisfying  ℓ≤ k, ℓ_1≤ k_1, ℓ_2≤ k_2.
  
  
  Note that the entries of V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i) admit the form χ_st(θ(d_i))= _s^*  Ξ(θ(d_i))  _t for s,t ∈𝖨(i). Putting these quadratic forms together with those in Lemma  <ref>, and modifying the definition of 𝒫 in (<ref>) by including those quadratic forms as entries of V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i), one can apply the proof strategy of Theorem <ref> mutatis mutandis to the proof of Proposition <ref>. Hence we omit the details and conclude the proof. 
  
  
  
  
  
  We can then propose an estimator of θ(d_i), that is ∑_t∈𝖨(i)μ_t /|𝖨(i)|.
  The estimator of θ(d_i) then leads to an estimator of d_i via taking the inverse θ^-1.  Applying Proposition <ref>, we see that this estimator admits the following expansion 
  
      1/|𝖨(i)|∑_t∈𝖨(i)μ_t = θ(d_i) - (d_i^2-y)θ(d_i) 1/|𝖨(i)|Tr  V_𝖨(i)^* Ξ(θ(d_i)) V_𝖨(i) +  O_≺(d_i^1/2δ_i0^1/2 N^-1/2 - ε).
   
  Therefore, to perform hypothesis testing with this estimator, we shall study the fluctuation of 1/|𝖨(i)|Tr  V_𝖨(i) ^* Ξ(θ(d_i)) V_𝖨(i) under appropriate scaling. Note that 1/|𝖨(i) |Tr  V_𝖨(i) ^* Ξ(θ(d_i)) V_𝖨(i) is simply a linear combination of the quadratic forms, i.e, 
  
      1/|𝖨(i) |Tr  V_𝖨(i) ^* Ξ(θ(d_i)) V_𝖨(i)= 1/|𝖨(i) |∑_t∈𝖨(i)χ_tt(θ(d_i)),
  
  where we recall χ_tt(z)= _t^*Ξ(z)_t. Hence,  the joint distribution of the above term and  the other quadratic forms in the expansion of eigenvector in Lemma  <ref> can be derived in the same manner as the proof of Theorem  <ref>.   
   
  Let  
      Φ_𝖨(i) :=-  √(N (d_i^2-y)) θ(d_i) 1/|𝖨(i)|∑_t∈𝖨(i)χ_tt(θ(d_i)).
  
  We have the following proposition.
  
    
  Under the assumptions in Theorem <ref>, we have 
  
      1/|𝖨(i)|∑_t∈𝖨(i)μ_t = θ(d_i) + √(d_i^2-y)/Φ_𝖨(i)+  O_≺(d_i^1/2δ_i0^1/2 N^-1/2 - ε),
  
   for some small fixed ε>0.  
   
   (1). For  any _k, k∈𝖨(i), we recall from (<ref>) with w=_k, 
   
      _k, P_𝖨(i)_k   =
       d_i^2-y/d_i(d_i+y)+ 
      1/√(N(d_i^2-y)) Θ__k^_k
      -1 /N∑_t∈𝖨^cd_id_t/(d_i-d_t)^2 (Π__t^_k)^2 
      
         + O_≺(N^-ε(N^-1/21/√(d_i^2-y) + 1/N∑_t∈{ i}^cd_id_t/(d_i-d_t)^2)).
   
  Further, (Φ_𝖨(i) , {Θ__k^_k}_k∈𝖨(i), {Π__t^_k}_k∈𝖨, t∈ (𝖨(i))^c) is asymptotically Gaussian distributed. In particular, 
   
      (Φ_𝖨(i) ,  {Θ__k^_k}_k∈𝖨(i))≃𝒩(0, C_𝖨(i) ,e)
   
   where C_𝖨(i) ,e is defined by the RHS of the following equations
   
      var (Φ_𝖨(i))≐2(1+d_i^-1)^2/|𝖨(i)|^2  + κ_4 (1-yd_i^-2)(1+d_i^-1)^2/|𝖨(i) |^2∑_k_1,k_2∈𝖨(i) s_2,2 (_k_1, _k_2),
         cov (Θ__k_1^_k_1, Θ__k_2^_k_2) ≐1_{k_1=k_2}·  2y 𝚑(d_i)^2 (1+ y 𝚑(d_i)^2) +  κ_4 (1-yd_i^-2) 𝚏(d_i)^2  s_2,2 (_k_1,_k_2), 
         cov(Φ_𝖨(i) , Θ__k^_k)≐2y 𝚑(d_i)^2 (1+d_i^-1) / |𝖨(i)|
      +κ_4(1-yd_i^-2)(1+d_i^-1) 𝚏(d_i)/|𝖨(i)|∑_t∈𝖨(i) s_2,2(_t, _k)
  
   for k,k_1,k_2∈𝖨(i).
   
   (2). Let 𝒥 be  any fixed index set. Let  {u_k}_k∈𝒥⊂Span{𝐯_j }_j∈ 1,M∖𝖨(i) be any given family of orthonormal vectors. Then according to the second case in Remark <ref>, i.e., (<ref>), we have 
   
      u_k, P_𝖨(i)u_k =
       ‖ς_𝖨(i)^u_k‖^2/Nd_i∑_t∈𝖨(i)(Δ__t^u_k)^2  
      +O_≺(N^-(1+ε)d_i^-1‖ς_𝖨(i)^u_k‖^2),
   
   with {Δ__t^u_k}_(k,t)∈𝒥×𝖨(i)≃𝒩(0, U_m). 
  
   Here U_m is a symmetric matrix of dimension |𝒥| |𝖨(i)| that can be defined by
   the RHS of the following equation
  
      cov (Δ__t_1^u_k_1, Δ__t_2^u_k_2) ≐   1_{t_1=t_2}𝚑(d_i) 
      ⟨(ς_t_1^u_k_1)^0, (ς_t_2^u_k_2)^0  ⟩
         + κ_4(d_i^2-y)/d_i^2 𝚑(d_i) s_1,1,1,1(_t_1, _t_2, (ς_t_1^u_k_1)^0, (ς_t_2^u_k_2)^0 ),
  
  for k_1,2∈𝒥, t_1,2∈𝖨(i). Here ( ς_𝖨(i)^u_k)^0  represents the normalized ς_𝖨(i)^u_k similarly to def.nor.Varsigma.
   
  
  
  
  
  
   First,  (<ref>) is an obvious conclusion of (<ref>)-(<ref>). Then, (<ref>) is simply a rewriting of (<ref>) with w=_k. Observe that Φ_𝖨(i)= |𝖨(i)|^-1∑_t∈𝖨(i)Φ_tt. In addition, according Remark <ref>, we can also write up the joint distribution of {Φ_tt}_t∈𝖨(i) and {Θ__k^_k}_k∈𝖨(i), which admits the same form as (<ref>) by choosing ℐ=𝖨(i).  Then the joint distribution in (<ref>) can be regarded as a corollary of such a more general joint distribution. 
  The whole statement (2) above is also analogous to its counterpart, Proposition <ref> (2); see Remark <ref>. The details can be checked similarly to Theorem <ref>, and are thus omitted. Hence, we conclude the proof. 
  
  
  
  
  
  With the  above propositions and remarks, we are now ready to prove Theorems <ref> and <ref>.
  
  
  
  
   
  Under the null  hypothesis of eq_spectralprojection, without loss of generality, we can choose  u_i= _i for i∈ℐ, since our statistic 𝒯 is independent of the choice of the basis of Z_0. To ease the understanding of the reader,  
  we will first show the proof in all-simple case, and then we discuss the all-equal case, and finally we conclude the extension to more general case. 
  
  First, assume that all d_i, i∈ℐ are simple. We assume that ξ_i is the eigenvector corresponding to  d_i and the direction _i.    By definition of 𝒯 in eq_teststat, we have the following derivation 
  
      𝒯   = ∑_i ∈ℐ⟨v_i, P_ℐv_i  ⟩-ϑ(d_i))
         = ∑_i ∈ℐ(|⟨ξ_i, v_i  ⟩|^2-ϑ( d_i) + ϑ( d_i)-ϑ(d_i))+ ∑_i ∈ℐ∑_j ∈ℐ∖{i}|⟨ξ_j, v_i  ⟩|^2
         = ∑_i ∈ℐ1/√(d_i^2-y)Θ__i^_i -  ∑_i∈ℐϑ'(d_i)(d_i- d_i) + O_≺(N^-ε∑_i ∈ℐ1/√(d_i^2-y)) 
         = ∑_i ∈ℐ1/√(d_i^2-y)Θ__i^_i -  ∑_i∈ℐϑ'(d_i)(θ^-1)'(θ(d_i)) √(d_i^2-y) Φ_i
             + O_≺(N^-ε∑_i ∈ℐ1/√(d_i^2-y)),
  
  for some small constant ε>0. Here in the third step, we used  Remark <ref> and Assumption <ref> to absorb the χ^2-terms into the error, and also the term ∑_i∈ℐ,  j ∈ℐ∖{i}|⟨ξ_j, v_i  ⟩|^2 which possesses the same bound that can be seen from Proposition <ref> and the definition of ς_i^u in (<ref>).  More precisely, from statement (2) of Proposition <ref>, we see that for each i∈ℐ, 
  
      ∑_  j ∈ℐ∖{i}|⟨ξ_j, v_i  ⟩|^2    = O_≺( ∑_j ∈ℐ∖{i}‖ς_j^_i‖^2 / d_j( Δ__j^_i)^2)
         = O_≺(  N^-1/2∑_j ∈ℐ, t∈{j}^c d_jd_t/(d_j-d_t)^2)
  
  where in the last step, we plugged in the definition of ς_i^u. Then by Assumption <ref>, we can also absorb the above term into the error.
  
  
   By elementary computation, we further obtain 
  
      ϑ'(d_i)= y(d_i^2+2d_i+y)/d_i^2(d_i+y)^2,    (θ^-1)'(θ(d_i))= (1- yd_i^-2)^-1,
  
  so that the coefficient of Φ_i  in the last line of (<ref>) is 
      -y(d_i^2+2d_i+y)/(d_i+y)^2(d_i^2-y)^1/2.
  
  Therefore, we can apply the result in Proposition <ref>  to finish the proof by the fact that  it is a linear combination of asymptotically Gaussian random variables.
  
  Next, we prove (<ref>) for the all-equal case, i.e. d_i=d_e for all i∈ℐ, and ℐ=𝖨(i). Recall the definition of 𝒯 in eq_teststat. We have 
  
      𝒯   = ∑_t∈ℐ(_t, P_ℐ_t - ϑ(d_t) +ϑ(d_t) - ϑ(d_t)) 
       
         = (d_e^2-y)^-1/2∑_t∈ℐΘ__t^_t  
       -   |ℐ|ϑ'(d_e) (θ^-1)'(θ(d_e)) (1/|ℐ|∑_i∈ℐμ_i- θ(d_e)) 
              + O_≺(N^-ε1/√(d_e^2-y)) 
         =  (d_e^2-y)^-1/2∑_t∈ℐΘ__t^_t  
       - |ℐ|ϑ'(d_e) (θ^-1)'(θ(d_e)) (d_e^2-y)^1/2Φ_ℐ
              + O_≺(N^-ε1/√(d_e^2-y))
  
   for some small constant ε>0.
   Note that by def:Phi_I and d_i=d_e for all i∈ℐ, we can rewrite Φ_𝖨(i)= 1/|𝖨(i)|∑_t∈ℐΦ_tt by setting Φ_tt:= -√(N(d_t^2-y))θ(d_t) χ_tt(θ(d_t)) which is consistent to the defined Φ_i≡Φ_ii in the simple  case (see Theorem <ref> and its proof). Hence, we can represent 𝒯 by 
   
      𝒯=     ∑_t∈ℐ (d_t^2-y)^-1/2Θ__t^_t  
       -  ∑_t∈ℐϑ'(d_t) (θ^-1)'(θ(d_t)) (d_t^2-y)^1/2Φ_tt
         + O_≺(N^-ε∑_t ∈ℐ1/√(d_t^2-y))
  
   which coincides with the  representation in 2020081601. And the asymptotic distribution is reduced to the joint distribution of {Θ__t^_t}_t∈ℐ, {Φ_tt}_t∈ℐ which is asymptotically Gaussian,  sharing the same covariance structure as all-simple case, as we mentioned in Remark <ref>. Since here d_i=d_e for all i∈ℐ, we can get the explicit expression of the  variance of 𝒯 (c.f.def.V1(d_I)) by applying (1) of  Proposition <ref>.
   
   In the end, we claim the result for the general case where ℐ= ⋃_k=1^ℓℐ_k such that all the spikes with indices from the same subset ℐ_k are equal and distinct otherwise with  distance satisfying Assumptions <ref> and <ref> . Especially, we do not further require that each subset is a singleton or not.  We then do the decomposition
   
      𝒯    = ∑_k=1^ ℓ∑_t∈ℐ_k(_t, P_ℐ_t - ϑ(d_t) +ϑ(d_t) - ϑ(d_t))  
         =  ∑_k=1^ ℓ(∑_t∈ℐ_k(_t, P_ℐ_k_t - ϑ(d_t) +ϑ(d_t) - ϑ(d_t)) ) 
             +  O_≺(N^-ε∑_t ∈ℐ1/√(d_t^2-y)).
  
   Applying the discussion for the all-equal case to each summand above, we can finish the proof analogously. 
   The details are omitted. Hence, we conclude the proof of Theorem <ref>. 
  
  
  
  
  
  Recall the definition of the statistic eq_finalstat. We can write
  
      N𝕋_2=N∑_j ∈𝒥⟨u_j, P_ℐu_j ⟩   =    ∑_k=1^ ℓ∑_j ∈𝒥 N ⟨u_j, P_ℐ_ku_j⟩
         =∑_i∈ℐ, j∈𝒥‖ς_i^_j‖^2/d_i (Δ__i^_j)^2 + O_≺(N^-ε∑_i ∈ℐ1/√(d_i^2-y)),
  
  for some small constant ε>0 by using  statement  (2) of Proposition <ref> for each summand in the third step. And the asymptotic distribution immediately follows from jointly CLT of {Δ__i^_j}_i∈ℐ, j∈𝒥. Then combining the results in
  Proposition <ref> and Proposition <ref> further with Remark <ref>, we can then finish the proof.
  
  
  
  
  
  
   In the end, we prove Proposition <ref>, by adapting the proof of Proposition 4.5 of <cit.>. We emphasize that the discussion in <cit.> was done for deformed Wigner matrices, where the strength of the deformation is required to be bounded above by a constant. Here we need to adapt the discussion to our model with possibly diverging d_i's. Hence, we need to keep tracking the dependence of the error terms on d_i more carefully. Nevertheless, apart from the size of d_i, the discussion is essentially the same as that of Proposition 4.5 of <cit.>.
   
  Similarly to the proof of Lemma <ref>,  μ_t is an eigenvalue outside the spectrum of H if and only if z=μ_t solves the equation det (D^-1+ zV^*_1(z)V)=0 or equivalently  D^-1+ μ_tV^*_1(μ_t)V has a zero eigenvalue. 
  By Lemma <ref>, we see that for t∈𝖨(i), |μ_t-θ(d_i)|= O_≺(d_i^1/2δ_i0^1/2N^-1/2).  Without loss of generality, up to a permutation, we assume the following decomposition
  
      D^-1= D_𝖨(i)^-1⊕ D_𝖨(i)^c^-1,    V= (V_𝖨(i), V_𝖨(i)^c).
  
  Here we recall 𝖨(i)^c= 1, r∖𝖨(i). 
   For simplicity, we set the following neighborhood of  θ(d_i)
  
      𝒟(θ(d_i))    = (θ(d_i)- d_i^1/2δ_i0^1/2N^-1/2 + δ, θ(d_i)+ d_i^1/2δ_i0^1/2N^-1/2 + δ) 
         =: (θ^-, θ^+ )
  
   for some sufficiently small δ. We will identify those μ_t's by analysing the behaviour of eigenvalues of D^-1+ xV^*_1(x)V as x varies. Our discussion in the sequel will rely on the fact  that the eigenvalues of D^-1+ xV^*_1(x)V should not attain zero simultaneously as x varies in 𝒟(θ(d_i)) with high probability, so that we can exactly find all the μ_t for t∈𝖨(i). In order to see this, similarly to the counterpart in <cit.>,  
  we introduce an additional randomness, by adding some small perturbation ιΔ where Δ is a Hermitian matrix with entries i.i.d. and has an absolutely continuous law supported in the unit disc. And the scalar ι≡ι(N)>0 can be chosen arbitrarily small, say e^-N. We now turn to  study the behaviours of eigenvalues of D^-1+ xV^*_1(x)V+ ιΔ instead. 
  
   First, for x∈𝒟(θ(d_i)) for some sufficiently small δ, we define two matrices 𝒜^ι(x) and 𝒜^ι(x)  by 
  
      𝒜^ι(x) := D^-1+ xV^*_1(x)V+ ιΔ- xm_1(x)I_r, 
         𝒜^ι(x):= 𝒜_𝖨(i)^ι(x) ⊕𝒜_𝖨(i)^c
      ^ι(x).
  
  Here we use the notation 𝒜^ι_I(x) to represent a submatrix of 𝒜^ι(x) by taking columns  and rows  both from I. In particular, 𝒜_𝖨(i)^ι(x)= D_𝖨(i)^-1+ xV^*_𝖨(i)_1(x)V_𝖨(i) + ιΔ_𝖨(i) - xm_1(x) I_|𝖨(i)|, where Δ_𝖨(i) is the submatrix of Δ with row and column indices both from I.  
  
  
  
  
  By definition, we see that 𝒜^ι(x) is obtained from 𝒜^ι(x) by taking the two  block matrices on diagonal.  Using isotropic local law est.DG and taking ι small enough, say e^-N, we can easily get 
  
      ‖𝒜^ι(x)- 𝒜^ι(x)‖_op≺  d_i^-1/2δ_i0^-1/2 N^-1/2.
  
  Next, we claim that the spectrum of the two block matrices in 𝒜^ι(x) are well separated, i.e. we will establish  a lower bound on the spectral gap 
  
      dist( Spec(𝒜_𝖨(i)^ι(x)), Spec(𝒜_𝖨(i)^c^ι(x))).
  
  This can be easily achieved by using triangle inequality and isotropic local law est.DG, by choosing ι sufficiently small, 
  
      dist( Spec(𝒜_𝖨(i)^c_l^ι(x)), Spec(𝒜_𝖨(i)^ι(x))) ≥dist ( D_𝖨(i)^-1, D_𝖨(i)^c^-1)  - d_i^-1/2δ_i0^-1/2 N^-1/2+ε/2
  
  with high probability. 
  Then by non-overlapping condition asd2, we see that dist ( D_𝖨(i)^-1, D_𝖨(i)^c^-1) > d_i^-1/2δ_i0^-1/2 N^-1/2 + ε for some small fixed ε>0. This implies with high probability, 
  
      dist(Spec(𝒜_𝖨(i)^ι(x)),Spec(𝒜_𝖨(i)^c^ι(x)))> d_i^-1/2δ_i0^-1/2 N^-1/2 + ε
  
  by slightly modifying the value of ε. 
  
  Further, we define { a_t^ι(x)}_t∈𝖨(i) (resp. { a_j^ι(x)}_j∈𝖨(i)^c) the eigenvalues of 𝒜_I^ι(x)  (resp. 𝒜_𝖨(i)^c^ι(x)) in an increasing order. And they are actually the eigenvalues of 𝒜^ι(x) due to its block structure. Correspondingly, we denote by a_k^ι(x) the eigenvalues of 𝒜^ι(x).  Note that for x∈𝒟(θ(d_i)), by isotropic local law est.DG
  
      a_t^ι(x) =  1+ d_i^-1 +O_≺(d_i^-1/2δ_i0^-1/2N^-1/2),  for  t∈𝖨(i)
          a_j^ι(x) =  1+ d_j^-1 +O_≺(d_i^-1/2δ_i0^-1/2N^-1/2),  for  j∈𝖨(i)^c.
  
  This with 2020073101 and non-overlapping condition asd2, implies that for j∈𝖨(i)^c,
  
      a_j^ι(x) + xm_1(x) ≍ d_j^-1-d_i^-1.
  
  Hence, a_j^ι(x) =- xm_1(x) get no solution on 𝒟(θ(d_i)) for j∈𝖨(i)^c.
  
  We then further set   H^ι:= Σ_ι^1/2 XX^* Σ_ι^1/2 where we define that Σ_ι=  I+ V (({1/d_t}_t∈𝖨(i), {1/d_j}_j∈𝖨(i)^c )+ ιΔ)^-1V^* with V= (V_𝖨(i), V_𝖨(i)^c).
  Note that since ι can be choosing arbitrarily small, then by Lemma <ref>, we get that H^ι has with high probability exactly |𝖨(i)| eigenvalues {μ^ι_t}_t∈𝖨(i) in 𝒟(θ(d_i)) and μ_t^ι -θ(d_i) = O_≺(d_i^1/2δ_i0^1/2N^-1/2) further with  μ_t^ι m_1(μ_t^ι)= -(1+1/d_i) +O_≺(d_i^-1/2δ_i0^-1/2N^-1/2). Here tentatively we  do not specify the ordering of the eigenvalues {μ^ι_t}_t∈𝖨(i). Later we will see how to  pin down each μ_t^ι for t∈𝖨(i).
  Recall the  notations in  note:2020080301. By elementary computation,
   det(D^-1+ μ_t^ι V^*_1(μ_t^ι)V+ ιΔ)=0 which implies that 𝒜^ι(μ_t^ι) has an eigenvalue -μ_t^ι m_1(μ_t^ι). 
   
    From all the above arguments, we see that all those μ_t^ι can  be a solution to the equation a_t^ι(x)=-xm_1(x) only when t∈𝖨(i) and x∈𝒟(θ(d_i)).  
  We can further claim that for all x∈𝒟(θ(d_i)),  a_t^ι(x)= -xm_1(x) for at most one t∈𝖨(i) (a.s.)
  by referring to  the proof of Proposition 4.5 in <cit.>, thanks to the continuously distributed perturbation term ιΔ.
  Next, we can claim that a_t^ι(x)= -xm_1(x) indeed has  solution in 𝒟(θ(d_i)) for each t∈𝖨(i) by the continuity of a_t^ι(x), -xm_1(x) and  showing that 
  
      a_t^ι(θ^-)< -θ^-m_1(θ^-),    a_t^ι(θ^+) > -θ^+m_1(θ^+).
  
  We will show the detail for the first inequality above and the second one can be done analogously.  By isotropic local law est.DG, we have 
  
      a_t^ι(θ^-) + θ^-m_1(θ^-)    =  a_t^ι(θ^-)+ θ^-m_1(θ^-) + O_≺(d_i^-1/2δ_i0^-1/2N^-1/2)
          = -θ(d_i)m_1(θ(d_i)) +θ^-m_1(θ^-)+ O_≺(d_i^-1/2δ_i0^-1/2N^-1/2) 
         = θ^- -θ(d_i)/d_i^2-y + O_≺(d_i^-1/2δ_i0^-1/2N^-1/2) 
         = - O_≺(d_i^-1/2δ_i0^-1/2N^-1/2+δ) <0.
  
  
  The above arguments state that  with high probability each equation  a_t^ι(x)= -xm_1(x) identifies at least one distinct μ_t^ι for t∈𝖨(i), and totally, we have |𝖨(i)| eigenvalues of H^ι  sitting in 𝒟(θ(d_i)). Thus with high probability, we shall have a one-to-one correspondence between  μ_t^ι and solution of  a_t^ι(x)= -xm_1(x) in 𝒟(θ(d_i)) for t∈𝖨(i) so that  
   -μ_t^ι m_1(μ_t^ι) = a_ t ^ι( μ_t^ι).
   Next, by isotropic local law est.DG, one can also check ‖∂𝒜^ι(x)‖_op≺ d_i^1/2δ_i0^-5/2N^-1/2  which together  with μ_t^ι -θ(d_i) = O_≺(d_i^1/2δ_i0^1/2N^-1/2) and asd further leads to 
   
      a_ t ^ι( μ_t^ι)-  a_ t ^ι( θ(d_i)) = O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε)
  
   for some small fixed ε>0.
   
   Furthermore, by using perturbation theory we can claim that 
   
      a_ t ^ι( θ(d_i)) =  a_ t ^ι( θ(d_i)) + O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε),
  
  which can be checked by applying Proposition A.1 of <cit.>  that 
  
      | a_ t ^ι( θ(d_i)) -  a_ t ^ι( θ(d_i)) | 
         ≺‖𝒜^ι - 𝒜^ι‖_op^2/dist( σ(𝒜_𝖨(i)^ι(x)), σ(𝒜_𝖨(i)^c^ι(x))) - ‖𝒜^ι - 𝒜^ι‖_op
         = O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε).
  
  Combining 2020080302, 2020080303 and  -μ_t^ι m_1(μ_t^ι) = a_ t ^ι( μ_t^ι), we eventually arrive at 
  
      -μ_t^ι m_1(μ_t^ι)     =  a_ t ^ι( θ(d_i)) + O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε) 
         = λ_ t (D_𝖨(i)^-1+ θ(d_i)V^*_𝖨(i)_1(θ(d_i))V_𝖨(i) + ιΔ_𝖨(i) - θ(d_i)m_1(θ(d_i)) I_|𝖨(i)|)
            +  O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε).
  
  Here we emphasize that we use the notation λ_t(A_𝖨(i)) for a |𝖨(i)|× |𝖨(i)| Hermitian matrix A_𝖨(i) indexed by |𝖨(i)| to denote its eigenvalues that admit the ordering  λ_t_1(A_𝖨(i))≤λ_t_2(A_𝖨(i)) for any t_1, t_2∈𝖨(i), t_1≤ t_2. Using the identity 1+ d_i^-1= - θ(d_i)m_1(θ(d_i)) and expanding μ_t^ι m_1(μ_t^ι) around θ(d_i)m_1(θ(d_i)),  we get 
  
      μ_t^ι - θ(d_i)   = -1/d_i^2-yλ_t( θ(d_i) V_𝖨(i)^* Ξ(θ(d_i))V_𝖨(i) + ιΔ_𝖨(i))
      
            +  O_≺(d_i^-1/2δ_i0^-1/2N^-1/2-ε).
  
  In the end, let ι→ 0, we ultimately proved  Proposition   <ref>.
  
   
   The conclusions in Corollaries <ref> and  <ref> follow from Theorems <ref>  and <ref> and the fact that d_ℐ (resp. d)  is a consistent estimator of 
  d_ℐ (resp. d) after appropriate scaling. 
  
   
   
  
  § ADDITIONAL SIMULATIONS
    
  In this section, we provide additional simulation results regarding the two-point random variable 1/3δ_√(2)+2/3δ_-1/√(2). 
  
  
  
   
  
  
  
  
  
  
  
  
  
  
   
  
  
  
  
  
  
  
  
  
  
    
   
   
  
  
   
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   
  
  
  
  
  
  
  
  unsrt
  1
  
  TA
  T. Anderson. An Introduction to Multivariate Statistical Analysis, 3rd edition. Wiley Series in Probability and Statistics, Wiley. 2003. 
  
  BN
  J. Bai, and S. Ng. Principal components estimation and identification of static factors. 
  Journal of Econometrics, 176(1):18–29, 2013. 
  
  BN2
  J. Bai, and S. Ng. Rank regularized estimation of approximate factor models. 
  Journal of Econometrics, 212(1):78–96, 2019.
  
  BN3
  J. Bai, and S. Ng. Determining the Number of Factors in Approximate Factor Models. 
  Econometrica, 70(1):191–221, 2002.
  
  BY08
  Z.D. Bai, J.F. Yao.
  Central limit theorems for eigenvalues in a spiked population model. 
  Annales de l'IHP Probabilités et statistiques 44(3), 447–474, 2008.
  
  BY12
  Z.D. Bai, J.F. Yao. 
  On sample eigenvalues in a generalized spiked population model. 
  Journal of Multivariate Analysis, 106, 167–177, 2012.
  
  BBP
  J. Baik,  G. Ben Arous, and  S. Péché.   
  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices.
  The Annals of Probability, 33(5), 1643–1697, 2005.
  
  BaikS
  J. Baik,  J.W. Silverstein.   
  Eigenvalues of large sample covariance matrices of spiked population models. 
  Journal of multivariate analysis, 97(6), 1382–1408, 2006.
  
  BDW
  Z.G. Bao, X.C. Ding, and K.  Wang.  
  Singular vector and singular subspace distribution for the matrix denoising model.
  The Annals of Statistics (in press), 2020. 
  
  BBCF
  S. Belinschi, H.  Bercovici, M.  Capitaine.  
  On the outlying eigenvalues of a polynomial in large independent random matrices. 
  arXiv:1703.08102, 2017.
  
  BBCFAOP
  S. Belinschi, H.  Bercovici, M.  Capitaine,  and M.  Février.  
  Outliers in the spectrum of large deformed unitarily invariant models. 
  The Annals of Probability, 45(6A), 3571-3625, 2017.
  
  BGM11
  F. Benaych-Georges, A. Guionnet, and M.  Maida.
  Fluctuations of the extreme eigenvalues of finite rank deformations of random matrices. 
  Electronic Journal of Probability, 16, 1621–1662, 2011.
  
  BGN12
  F. Benaych-Georges, R. R.  Nadakuditi.  
  The singular values and vectors of low rank perturbations of large rectangular random matrices. 
  Journal of Multivariate Analysis, 111, 120–135, 2012.
  
  BGN11
  F. Benaych-Georges, R. R.  Nadakuditi.   
  The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. 
  Advances in Mathematics, 227(1), 494–521, 2011.
  
  BJNP
  A. Birnbaum, I. Johnstone, B. Nadler, and D. Paul.
  Minimax bounds for sparse PCA with noisy high-dimensional data.
  Ann. Stat., 41(3):1055–1084,2013. 
  
  
  
  bloemendal2016principal
  A. Bloemendal, A. Knowles, H.-T. Yau, and J. Yin.
  On the principal components of sample covariance matrices.
  Probability theory and related fields, 164(1-2):459–552, 2016.
  
  BVirag13
  A. Bloemendal, B. Virág.   
  Limits of spiked random matrices I. 
  Probability Theory and Related Fields, 156(3-4), 795–825, 2013.
  
  BVirag16
  A. Bloemendal, B. Virág. 
  Limits of spiked random matrices II. 
  The Annals of Probability, 44(4), 2726–2769, 2016.
  
  bloemendal2014isotropic
  A. Bloemendal, L. Erdos, A. Knowles, H.-T. Yau, and J. Yin.
  Isotropic local laws for sample covariance and generalized Wigner
    matrices.
  Electron. J. Probab, 19(33):1–53, 2014.
  
  BBPA
  J. Bun, J. Bouchaud, and M. Potters. 
  Cleaning large correlation matrices: Tools from Random Matrix Theory. Physics Reports, 666:1-109, 2017.
  
  Capitaine17
  M. Capitaine.  
  Limiting eigenvectors of outliers for Spiked Information-Plus-Noise type matrices. 
  Séminaire de Probabilités XLIX 119–164, Springer, Cham, 2018.
  
  CDM16
  M. Capitaine, C. Donati-Martin. 
  Spectrum of deformed random matrices and free probability. 
  arXiv:1607.05560, 2016.
  
  CDM18
  M. Capitaine, C. Donati-Martin. 
   Non universality of fluctuations of outlier eigenvectors for block diagonal deformations of Wigner matrices. 
   arXiv:1807.07773, 2018.
  
  CDF09
  M. Capitaine, C. Donati-Martin, and D. Féral.  
  The largest eigenvalues of finite rank deformation of large Wigner matrices: convergence and nonuniversality of the fluctuations. 
  The Annals of Probability, 37(1), 1–47, 2009.
  
  CDF12
  M. Capitaine, C. Donati-Martin, and D. Féral.  
   Central limit theorems for eigenvalues of deformations of Wigner matrices. 
   Annales de l'IHP Probabilités et statistiques  48(1),  107–133, 2012.
   
  
   
   Ding
   X.C. Ding.  
   High dimensional deformed rectangular matrices with applications in matrix denoising.
   Bernoulli, 26(1):387–417,2020. 
   
   DY19
    X.C. Ding, F. Yang.
   Spiked separable covariance matrices and principal components.
   The Annals of Statistics (to appear), 2020.
   
   DO
   E. Dobriban and A. Owen.
   Deterministic parallel analysis: an improved method for selecting factors and principal components.
   , 81(1):163-183, 2019.
   
    
   ED
   E. Dobriban. 
   Permutation methods for factor analysis and PCA. 
   The Annals of Statistics(to appear), 2020. 
   
   shrinkage
  D. Donoho, M. Gavish, and I. Johnstone. Optimal shrinkage of eigenvalues in the spiked covariance model.The Annals of Statistics, 46(4): 1742–1778, 2018. 
   
  
  
  EKY2013
  L. Erdős, A. Knowles, and H.-T. Yau.
  Averaging fluctuations in resolvents of random band matrices.
  Ann. Henri Poincaré, 14(8):1837–1926, 2013.
  
  FFHL
  J. Fan, Y. Fan, X. Han, and  J.  Lv.  
  Asymptotic theory of eigenvectors for large random matrices. 
  arXiv:1902.06846, 2019.
  
  FLM
  J. Fan, Y. Liao, and M. Mincheva. 
  High-dimensional covariance matrix estimation in approximate factor models. 
  Ann. Stat., 39(6):3320–3356, 2011. 
  
  FP07
  D. Féral, S. Péché. 
  The largest eigenvalue of rank one deformation of large Wigner matrices. 
  Communications in mathematical physics, 272(1), 185–228, 2007.
  
  LeCamtest
  M. Hallin, D. Paindaveine and T. Verdebout. Optimal rank-based testing for principal components. The Annals of Statistics, 38(6), 3245–3299, 2010.  
  
  HK
  Y. He,  A. Knowles.
  Mesoscopic eigenvalue statistics of Wigner matrices. 
  The Annals of Applied Probability, 27(3), 1510-1550, 2017.
  
  
  HK2
  Y. He,  A. Knowles.
  Mesoscopic eigenvalue density correlations of Wigner matrices.
  Probability Theory and Related Fields, 1-70, 2019.
  
  HLS
  J. Hwang, J. Lee, and K. Schnelli. 
  Local law and Tracy–Widom limit for sparse sample covariance matrices.
  Ann. Appl. Probab., 29(5):3006–3036, 2019.
  
  
  JWHT
  G. James, D. Witten, T. Hastie, and R. Tibshirani. 
  An Introduction to Statistical Learning: with Applications in R. Springer Texts in Statistics , Springer. 2013. 
  
  Johnstone
  I.  Johnstone. 
  On the distribution of the largest eigenvalue in principal components analysis. 
  The Annals of statistics, 29(2): 295–327,  2001.
  
  sparse1
  I. Johnstone, and A. Lu.
  On Consistency and Sparsity for Principal Components Analysis in High Dimensions.
  Journal of American Statistical Association, 104(486):682–693, 2009.
  
  JY
  I. Johnstone, and  J. Yang.  
  Notes on asymptotics of sample eigenstructure for spiked covariance models with non-Gaussian data. 
  arXiv:1810.10427, 2018.
  
  JI
  I. T. Jolliffe. Principal Component Analysis. Springer Series in Statistics, Springer-Verlag New York, 2002. 
  
  KML
  Z. Ke, Y. Ma, and X. Lin. Estimation of the number of spiked eigenvalues in a
  covariance matrix by bulk eigenvalue matching analysis. arXiv preprint arXiv: 2006.00436, 2020. 
  
  KKP96
  A.M.  Khorunzhy, B.A. Khoruzhenko, and L.A.  Pastur.
  Asymptotic properties of large random matrices with independent
    entries.
  Journal of Mathematical Physics, 37(10):5033–5060, 1996.
  
  
  KY13
  A. Knowles,  J. Yin. 
  The isotropic semicircle law and deformation of Wigner matrices. 
  Communications on Pure and Applied Mathematics, 66(11), 1663–1749, 2013. 
  
  KY14
  A. Knowles,  J. Yin. 
  The outliers of a deformed Wigner matrix. 
  The Annals of Probability, 42(5), 1980–2031, 2014.
  
  knowles2017anisotropic
  A. Knowles,  J. Yin.
  Anisotropic local laws for random matrices.
  Probability Theory and Related Fields, 169(1-2):257–352, 2017.
  
  KLsplit
  V. Koltchinskii and K. Lounici. New asymptotic results in Principal Component Analysis.  Sankhya A., 79(2), 254-297, 2017.
  
  loadinginterval1
  Z.V. Lambert, A.R. Wildt, and R. M. Durand. Approximating Confidence Intervals for Factor Loadings.  Multivariate Behavioral Research,  26(3):421-434, 1991. 
  
  LS18
  J. O. Lee,  K.  Schnelli.   
  Local law and Tracy-Widom limit for sparse random matrices. 
  Probability Theory and Related Fields, 171(1-2), 543-616, 2018.
  
  LHY
  Z. Li, F. Han, and J. Yao. 
  Asymptotic joint distribution of extreme eigenvalues and trace of large sample covariance matrix in a generalized spiked population model.
  The Annals of Statistics (in press), 2020.  
  
  LV
  P. Loubaton,  P.  Vallet.  
  Almost sure localization of the eigenvalues in a Gaussian information plus noise model. Application to the spiked models. 
  Electronic Journal of Probability, 16, 1934–1959, 2011.
  
  LP09
  A. Lytova and L. Pastur.
  Central limit theorem for linear eigenvalue statistics of random
    matrices with independent entries.
  Ann. Probab., 37(5):1778–1840, 2009.
  
  MP67
  V.A. Marčenko, L.A. Pastur.
  Distribution of eigenvalues for some sets of random matrices.
  Mathematics of the USSR-Sbornik, 1(4):457, 1967.
  
  sparse2
  Z. Ma.
  Sparse principal component analysis and iterative thresholding. 
  Ann. Stat., 41(2):772–801, 2013. 
  
  Monasson_2015
  R. Monasson, and D. Villamaina. Estimating the principal components of correlation matrices from all their empirical eigenvectors. 
  Europhysics Letters, 112(5): 50001, 2015. 
  
  MJMY
  D. Morales-Jimenez, I. M. Johnstone, M. R. McKay, and J.  Yang.   
  Asymptotics of eigenstructure of sample correlation matrices for high-dimensional spiked models. 
  arXiv:1810.10214, 2018.
  
  NE2008
  R. R. Nadakuditi, and A., Edelman.  
  Sample eigenvalue based detection of high-dimensional signals in white noise using relatively few samples. 
  IEEE Transactions on Signal Processing, 56(7), 2625-2638, 2008.
  
  NS2010
  R. R. Nadakuditi, and J. W.  Silverstein. 
  Fundamental limit of sample generalized eigenvalue based detection of signals in noise using relatively few signal-bearing and noise-only samples. 
  IEEE Journal of Selected Topics in Signal Processing, 4(3), 468-480, 2010.
  
  Nadler2008
  B. Nadler.  
  Finite sample approximation results for principal component analysis: A matrix perturbation approach. 
  The Annals of Statistics, 36(6), 2791-2817, 2008.
  
  
  NJ
  B. Nadler and I. M. Johnstone.
  Detection performance of Roy's largest root test when the noise covariance matrix is arbitrary.
  2011 IEEE Statistical Signal Processing Workshop (SSP), 681–684, 2011.
  
  bootsfr
  A. Naumov, V. Spokoiny, and V. Ulyanov.  
  Bootstrap confidence sets for spectral projectors of sample covariance. 
  Probability Theory and Related Fields, 1091–1132, 2019.
  
  ONA
  A. Onatski. Testing Hypotheses About the Number of Factors in Large Factor Models. 
  Econometrica, 77(5):1447–1479,2009.
  
  
  
  
  ONA1
  A. Onatski, M. Moreira, and M. Hallin. Signal detection in high dimension: The multispiked case. Ann. Statist., 42(1): 225-254, 2014. 
  
  
  ONA2
  A. Onatski. Asymptotics of the principal components estimator of large factor models with weakly influential factors. Journal of Econometrics, 168(2):244-258, 2012. 
   
  
  loadinginterval2
  F.J. Oort. Likelihood-Based Confidence Intervals in Exploratory Factor Analysis. Structural Equation Modeling: A Multidisciplinary Journal, 18(3):383-396, 2011. 
  
  PYspike
  D. Passemier, J.F. Yao. Estimation of the number of spikes, possibly equal, in the high-dimensional case. Journal of Multivariate Analysis, 127: 173-183, 2014. 
  
  
  Paul
  D. Paul.
  Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. 
  Statistica Sinica, 1617–1642, 2007.
  
  Peche06
  S. Péché.
  The largest eigenvalue of small rank perturbations of Hermitian random matrices. 
  Probability Theory and Related Fields, 134(1), 127–173, 2006.
  
  PWBM
  A. Perry, A.S. Wein, A.S. Bandeira, A. Moitra. 
  Optimality and sub-optimality of PCA I: Spiked random matrix models. 
  The Annals of Statistics, 46(5), 2416-51, 2018.
  
  PY14
   N. S. Pillai, J. Yin.
  Universality of covariance matrices. 
  The Annals of Applied Probability, 24(3), 935-1001,  2014.
  
  PRS13
  A. Pizzo, D. Renfrew, and A. Soshnikov. 
  On finite rank deformations of Wigner matrices. 
  Annales de l'IHP Probabilités et statistiques 49(1),  64–94, 2013.
  
  MP
  M. Pourahmadi.
  High-Dimensional Covariance Estimation: With High-Dimensional Data. Wiley, 2013. 
  
  SZ
  H. Shen, and J. Huang. 
  Sparse principal component analysis via regularized low rank matrix approximation. 
  Journal of Multivariate Analysis, 99(6):1015–1034, 2008. 
  
  
  SF
  I. Silin, and J. Fan. 
  Hypothesis testing for eigenspaces of covariance matrix. 
  arXiv: 2020.09810, 2020.
  
  SS
  I. Silin, and V. Spokoiny. Bayesian inference for spectral projectors of the covariance matrix. Electronic Journal of Statistics, 12, 1948-1987, 2018. 
  
  tyler1981
  D.E.Tyler. Asymptotic inference for eigenvectors. The Annals of Statistics, 9, 725–736,1981. 
  
  tyler1983
  D.E.Tyler. A class of asymptotic tests for principal component vectors. The Annals of Statistics, 11, 1243–1250,1983. 
  
  XYY
  H.K. Xi, F. Yang, J. Yin.  Convergence of eigenvector empirical spectral distribution of sample covariance matrices. Annals of Statistics, 48(2):953-82, 2020.
  
  loadingtest
  H. Yamamoto, T. Fujimori, H. Sato, G. Ishikawa, K. Kami, and Y. Ohashi. Statistical hypothesis testing of factor loading in principal component analysis and its application to metabolite set enrichment analysis. BMC Bioinformatics, 15:51, 2014. 
  
  YZB
  J. Yao, S. Zheng, and Z. Bai. Large Sample Covariance Matrices and High-Dimensional Data Analysis. Cambridge University Press, 2015. 
  
  ZHT
  H. Zou, T. Hastie, and R. Tibshirani. Sparse Principal Component Analysis. Journal of Computational and Graphical Statistics, 15(2): 265–286, 2006. 
  
  
  
  
  
  