Optimal Eigenvalue Shrinkage in the Semicircle Limit

Abstract: Modern datasets are trending towards ever higher dimension. In response, recent theoretical studies of covariance estimation often assume the proportional-growth asymptotic framework, where the sample size $n$ and dimension $p$ are comparable, with $n, p \rightarrow \infty $ and $\gamma_n = p/n \rightarrow \gamma > 0$. Yet, many datasets -- perhaps most -- have very different numbers of rows and columns. We consider instead the disproportional-growth asymptotic framework, where $n, p \rightarrow \infty$ and $\gamma_n \rightarrow 0$ or $\gamma_n \rightarrow \infty$. Either disproportional limit induces novel behavior unseen within previous proportional and fixed-$p$ analyses. We study the spiked covariance model, with theoretical covariance a low-rank perturbation of the identity. For each of 15 different loss functions, we exhibit in closed form new optimal shrinkage and thresholding rules. Our optimal procedures demand extensive eigenvalue shrinkage and offer substantial performance benefits over the standard empirical covariance estimator.Practitioners may ask whether to view their data as arising within (and apply the procedures of) the proportional or disproportional frameworks. Conveniently, it is possible to remain {\it framework agnostic}: one unified set of closed-form shrinkage rules (depending only on the aspect ratio $\gamma_n$ of the given data) offers full asymptotic optimality under either framework. At the heart of the phenomena we explore is the spiked Wigner model, in which a low-rank matrix is perturbed by symmetric noise. Exploiting a connection to the spiked covariance model as $\gamma_n \rightarrow 0$, we derive optimal eigenvalue shrinkage rules for estimation of the low-rank component, of independent and fundamental interest.

\section{Introduction} \label{sec:1}

Suppose we observe $p$-dimensional Gaussian vectors
$x_1, \ldots, x_n \stackrel{i.i.d.}{\sim} \mathcal{N}(0, \Sigma)$, with
$\Sigma = \Sigma_{p}$ the underlying $p$-by-$p$ population covariance matrix.
Traditionally, to estimate $\Sigma$, we form the empirical (sample)
covariance matrix $S = S_n = \frac{1}{n} \sum_{i=1}^n x_i x_i'$;
this is the maximum likelihood estimator.  Under the classical
asymptotic framework that statisticians have used
for centuries, where $p$ is fixed and $n \rightarrow \infty$,
$S$ is a consistent estimator of $\Sigma$ (under any matrix norm).

In recent decades,  many impressive random matrix-theoretic studies consider $p = p_n$ tending to infinity with $n$. Generally, these studies focus on {\it proportional growth}, where the sample size and dimension
are comparable:
 \begin{align} n, p \rightarrow \infty \, , \qquad \gamma_n = \frac{p}{n} \rightarrow \gamma > 0 \, . \label{pro}
 \end{align}
 Under this framework, certain beautiful and striking mathematical phenomena are elegantly brought to light.
A striking deliverable for statisticians, particularly, is the discovery that
in such a high-dimensional setting, the maximum likelihood estimator $S$ is
an inconsistent estimator of $\Sigma$ (under various matrix norms).

\subsection{Standard Covariance Estimation in the Proportional Regime}

Inconsistency of $S$ under proportional growth stems from the following phenomena, not present in  classical fixed-$p$ statistics.  These results are due to  Marchenko and Pastur \cite{MP67}, Baik, Ben Arous, and P\'ech\'e \cite{BBP}, Baik and Silverstein \cite{BkS04}, and Paul \cite{P07}.

\begin{enumerate}

\item {\it Eigenvalue spreading}. Assume proportional growth with $\gamma \in (0, 1]$. In the standard normal case $\Sigma = I_{p}$, the $p$-dimensional identity matrix, the  spectral measure of $S$ converges weakly almost surely to the Marchenko-Pastur distribution with parameter $\gamma$. This  distribution, or \textit{bulk}, is  non-degenerate, absolutely continuous, and has support $[(1-\sqrt{\gamma})^2,  (1+\sqrt{\gamma})^2] = [\lambda_-(\gamma),\lambda_+(\gamma)] $.

Intuitively, sample eigenvalues, rather than concentrating near
 population eigenvalues (which, in the identity case, are all  simply $1$),
spread out across a fixed-size interval, preventing consistency of $S$ for $\Sigma$.

\item {\it Eigenvalue inflation}.
We can quantify asymptotic bias in extreme eigenvalues more precisely.
Consider Johnstone's {\it spiked covariance model},
 where all but finitely many eigenvalues $\ell_1, \ldots, \ell_{p}$ of $\Sigma$ are identity:
\begin{align*} \hspace{-1cm} [\textbf{I}]\hspace{1.2cm} \ell_1 > \cdots > \ell_r > 1\, , \qquad \ell_{r+1} = \cdots = \ell_{p} = 1 \, .   \end{align*}
Under this model, the number of so-called ``spiked'' eigenvalues $r$ is fixed and independent of $n$. Spiked eigenvalues are constant.

Let $\lambda_{i} = \lambda_{i, n}$
denote the eigenvalues of $S$,
ordered decreasingly
$\lambda_1  \geq \cdots \geq \lambda_{p}$.
As it turns out,  $\lambda_1, \ldots, \lambda_r$---the ``leading'' eigenvalues---do not converge to their population counterparts $\ell_1, \ldots, \ell_r$;
rather, they are shifted upwards. Under (\ref{pro}), for each fixed $i \geq 1$,
\begin{align} \phantom{\,.}     \lambda_i \xrightarrow{a.s.}   \lambda(\ell_i) \, ,
\label{1} \end{align}
where  $\lambda(\ell) \equiv \lambda(\ell, \gamma)$ is the
``eigenvalue mapping" function, given piecewise by
\begin{align}  \lambda(\ell)  = \begin{dcases}
      \ell + \frac{\gamma \ell}{\ell-1} & \ell > 1 + \sqrt{\gamma}\\
      (1 + \sqrt{\gamma})^2 & \ell \leq 1 + \sqrt{\gamma}
   \end{dcases} \, .    \label{bias_func}
\end{align}

The transition point $\ell_+(\gamma) \equiv 1 + \sqrt{\gamma}$
between the two behaviors is known as the Baik-Ben Arous-P\'ech\'e (BBP) transition.
Below the transition,
$1 < \ell \leq \ell_+(\gamma)$, ``weak signal" leads to a limiting eigenvalue independent of $\ell$. For fixed $i$ such that $\ell_i \leq \ell_+(\gamma)$, $\lambda_i$ tends to $\lambda_+(\gamma)$, the upper-bulk-edge of the Marchenko-Pastur distribution with parameter $\gamma$.

Above the transition,
$ \ell_+(\gamma) < \ell$, ``strong signal" produces an empirical eigenvalue dependant on $\ell$, though with upwards bias:
$\lambda(\ell) > \ell$. This asymptotic bias in extreme eigenvalues
is a further cause of inconsistency of $S$ in several loss measures,
 obviously including operator norm.

\item {\it Eigenvector rotation.}
The eigenvectors $v_1, \ldots, v_{p}$ of $S_n$
do not align asymptotically with the
corresponding eigenvectors $u_1, \ldots, u_{p}$ of $\Sigma$.
Under  proportional growth,
the limiting angles are deterministic, and obey:
\begin{align} | \langle u_i, v_j \rangle |    \xrightarrow{a.s.}  \delta_{ij} \cdot c(\ell_i) \, , \hspace{2cm}  1 \leq i, j \leq r \, ; \label{4}
\end{align}
here the ``cosine" function $c(\ell) = c(\ell, \gamma)$
is given piecewise by
\begin{align} \phantom{\,.}  c^2(\ell)  = \begin{dcases}
      \frac{1 - \gamma/(\ell-1)^2}{1+\gamma/(\ell-1)}  & \ell > 1 + \sqrt{\gamma}\\
      0 & \ell \leq 1 + \sqrt{\gamma}
   \end{dcases} \, .  \label{2} \end{align}
\noindent  Again, a phase transition occurs at $\ell_+(\gamma)$.
The misalignment of empirical and theoretical eigenvectors further contributes to
inconsistency in the spiked case where not all population eigenvalues are equal;
this is easiest to see for Frobenius loss.

\end{enumerate}

\subsection{Shrinkage Estimation}

Charles Stein proposed {\it eigenvalue shrinkage}
as an alternative to traditional covariance estimation  \cite{S86, S56}.
Let $S = V \Lambda V'$ denote an eigendecomposition,
where $V$ is orthogonal and $\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_{p})$.
Let $\eta: [0, \infty) \rightarrow [0, \infty)$ denote a ``shrinkage" function or ``rule" and $\eta(\Lambda) = \text{diag}(\eta(\lambda_1), \ldots, \eta(\lambda_{p}))$.
Estimators of the form  $\widehat \Sigma_\eta = V \eta(\Lambda) V'$
are studied in hundreds of papers;  see for example work of Donoho, Gavish, and Johnstone \cite{DGJ18} and Ledoit and Wolf \cite{LW20, LW202}. Note that while there is indeterminism in the choice of eigenvectors $V$, $\widehat \Sigma_\eta$ is well defined.

The standard sample covariance estimator $S$ is obtained by
the identity ``shrinker'' $\eta(\lambda) = \lambda$, no shrinkage at all,
while ``effective'' shrinkers generally act as contractions, obeying $|\eta(\lambda) - 1|  <  |\lambda - 1|$.
In the spiked model, a well-chosen shrinker
minimizes the errors induced by eigenvalue spreading and eigenvector rotation.
Working under the spiked model and proportional growth,
\cite{DGJ18} considers numerous loss functions $L$
and derives asymptotically unique admissible shrinkers  $\eta^*( \cdot | L)$, in many cases far outperforming $S$.

\subsection{Which Choice of Asymptotic Framework?}

The modern ``big data" explosion exhibits all manner of ratios of dimension to sample size.
Indeed, there are internet traffic datasets with billions of samples
and thousands of dimensions, and computational
biology datasets with thousands of samples and millions of dimensions.
To consider only asymptotic frameworks where row and column counts are roughly balanced,
as they are under proportional growth, is a restriction, and
perhaps, even an obstacle.

Although proportional growth analysis has yielded
many valuable insights, it also raises pressing
doubts in applications.
Consider the practitioner's ``scaling conundrum'':
in a given application, with one dataset
of size $(n_\text{data}, p_\text{data})$,
how can a practitioner know whether the proportional growth model is applicable?

Implicit in the choice of asymptotic framework is an assumption on how the data arises in a sequence of growing datasets; this choice has consequences. The practitioner may view the data as arising from the fixed-$p$ sequence
$(n,p_{\text{data}})$ with only $n$ varying.
If so,  the practitioner could appeal to long tradition and estimate $\Sigma$ by $S$.
On the other hand,  viewing
the dataset size as part of a sequence $\big(n,\frac{p_{\text{data}}}{n_{\text{data}}} \cdot n\big)$,
with constant aspect ratio $\gamma = p_\text{data} / n_\text{data}$, the practitioner might follow recent trends in the theoretical literature and apply eigenvalue shrinkage.

Current theory offers little guidance on choice which perspective is more appropriate, particularly if
$p_\text{data}$ is large yet relatively small compared to $n_\text{data}$.
And---importantly---these two are not the only possible scaling relations
that could produce the given $(n_\text{data}, p_\text{data})$.

\subsection{Disproportional Growth}

Within the full spectrum of power law scalings $p \asymp n^\alpha$, $\alpha \geq 0$,
the much-studied proportional growth limit corresponds to the {\it single
case} $\alpha=1$. The classical $p$-fixed, $n$ growing relation again
corresponds to a single
case, $\alpha=0$.
This paper considers {\it disproportional growth},
encompassing
{\it everything else}:
\[ n, p \rightarrow \infty \, , \qquad \gamma_n = p/n \rightarrow 0  \text{ or } \infty \, .
\]
Note that all power law scalings $0 < \alpha < \infty$, $\alpha \neq 1$ are included, as well as non-power law scalings, such as\ $p = \log n$ or $p = e^n$. The disproportional growth framework splits naturally into instances;
to describe them, we use terminology that assumes the underlying data matrices
$X = X_n$ are $p \times n$.

\begin{enumerate}

\item The ``wide matrix''  disproportional limit obeys:
\begin{equation} \label{tall}
    n, p \rightarrow \infty \, , \qquad \gamma_n = p/n \rightarrow  0 .
\end{equation}
In this limit, which includes power laws with $\alpha \in (0,1)$, $n$ is much larger than $p$, and yet we are
outside the classical, fixed-$p$ large-$n$ setting.

\item  The ``tall matrix'' disproportional limit  involves
arrays with many more columns than rows; formally:
\begin{equation} \label{wide}
   n, p \rightarrow \infty \, , \qquad \gamma_n = p/n \rightarrow \infty \, .
\end{equation}
This limit, including power laws with $\alpha \in (1, \infty)$, admits many additional scalings
of numbers of rows to columns.
\end{enumerate}

Properties of covariance matrices in the two disproportionate limits are closely linked.
Indeed,
the non-zero eigenvalues of $XX'$ and $X'X$ are equal. In the standard normal case $\Sigma = I$, $XX'$ and $X'X$
are the unnormalized sample covariance
matrices of wide and tall datasets, respectively.
Elaborating this, for any sequence of tall matrices with $\gamma_n \rightarrow \infty$, there is an accompanying sequence of wide matrices with  $\gamma_n \rightarrow 0$ and related spectral properties.

\subsection{Estimation as $\gamma_n \rightarrow 0$}

The $\gamma_n \rightarrow 0$ regime seems, at first glance, very different from
the proportional case, $\gamma_n \rightarrow \gamma > 0$. Neither eigenvalue spreading nor eigenvalue inflation are apparent: under spikes {\bf I}, empirical eigenvalues converge to their population counterparts: $\lambda_i \rightarrow \ell_i$, almost surely. Eigenvalue shrinkage may therefore seem irrelevant or unhelpful: $S$ itself is a consistent estimator of $\Sigma $ in Frobenius and operator norms. To the contrary, we will show that well-designed eigenvalue shrinkage confers substantial relative gains over standard covariance estimation, paralleling gains seen earlier under proportional growth.

Eigenvalue spreading {\it does} occur as $\gamma_n \rightarrow 0$, though only by $O(\sqrt{\gamma_n})$. Accordingly, introduce the quantities
\[ \tlam = \frac{\lambda -1}{\sqrt{\gamma_n}} \, , \qquad \,\,\, \overset{\leftharpoonup}{\ell} = \frac{\ell - 1 }{\sqrt{\gamma_n}} \, , \]
measuring
leading empirical and population eigenvalues on a finer scale.
We adapt the spiked covariance model to the $\gamma_n \rightarrow 0$
setting, with spiked eigenvalues $\ell_i = \ell_{i;n} = 1 + \overset{\leftharpoonup}{\ell}_i \sqrt{\gamma_n}$
varying with $n$ while maintaining $\overset{\leftharpoonup}{\ell}_i$ fixed. On this scale,  as we shall see, in addition to eigenvalue spreading, eigenvalue inflation and eigenvector rotation occur as well. The consequences of such high-dimensional phenomena are similar to those uncovered in the proportional setting.
For many choices of loss function,
$S$ is outperformed substantially
by well-designed shrinkage functions, particularly near
the phase transition at $\ell_+(\gamma_n)$.
We will consider a range of
loss functions $L$, deriving for each
a  shrinker $\eta^*( \cdot | L)$ which is optimal as
$\gamma_n \rightarrow 0$. Analogous results hold as $\gamma_n \rightarrow \infty$.

\subsection{Estimation in the Spiked Wigner Model}

At the heart of our analysis is a connection to the {\it spiked Wigner model}. Let $W = W_n$ denote a {\it Wigner matrix}, a real symmetric matrix of size $n \times n$ with independent entries on the upper triangle
distributed as $\mathcal{N}(0,1)$. Let  $\Theta = \Theta_n$ denote a symmetric $n \times n$ ``signal'' matrix of fixed rank $r$;
 under the {\it spiked Wigner model} observed data  $Y = Y_n$ obeys
\begin{align}
\phantom{\,. }Y = \Theta + \frac{1}{\sqrt{n}} W     \,. \label{model2}
\end{align}

Let $\theta_1 \geq \cdots \geq \theta_{r_+} > 0 > \theta_{r_+ + 1} \geq \dots  \geq \theta_{r}$
denote the non-zero eigenvalues of $\Theta$,
so there are $r_+$  positive values and $r_- = r-r_+$ negative.

A standard approach to recovering $\Theta$
from noisy data $Y$ uses the eigenvalues of $Y$,
$\lambda_1(Y) \geq \cdots \geq \lambda_n(Y)$,
and the associated eigenvectors $v_1, \ldots, v_n$:
\[
\widehat \Theta^{r} = \sum_{i=1}^{r_+}  \lambda_i(Y) v_i v_i'  +  \sum_{i=n-r_-+1}^n  \lambda_i(Y) v_i v_i' \, .
\]
As it turns out, $\widehat \Theta^r$ can be improved upon substantially by
estimators of the form
\begin{align}
    \phantom{\,.}  \widehat\Theta_\eta  = \sum_{i=1}^n \eta(\lambda_i(Y)) v_i v_i' \, ,
\end{align}
with $\eta: \mathbb{R}^+ \rightarrow \mathbb{R}^+$ a well-chosen shrinkage function.

Optimal shrinkage formulas for the spiked Wigner model are equivalent  to  optimal formulas for covariance estimation in the disproportionate, $\gamma_n \rightarrow 0$ limit.  This is not mere coincidence.
As $\gamma_n \rightarrow 0$,
the spectral measure of $\gamma_n^{-1/2}(S - I_{p})$
converges (weakly almost surely)
to the semicircle law
(Bai and Yin \cite{BY88}); this is precisely the limiting spectral measure of spiked Wigners $Y$. Moreover, we show below that the driving
theoretical quantities in each setting---leading eigenvalue inflation, eigenvector rotation,
optimal shrinkers, and losses---are all ``isomorphic.''

\subsection{Our Contributions} \label{1.7}

Given this background, we now state our contributions:
\begin{enumerate}

\item We study the disproportionate $\gamma_n \rightarrow 0$ limit
with an eye towards establishing and using analogues of (\ref{1})-(\ref{2}). In the critical scaling  of  this regime, spiked eigenvalues decay towards one as $1 +  \overset{\leftharpoonup}{\ell} \sqrt{\gamma_n}$, where $\overset{\leftharpoonup}{\ell}$ is a new formal parameter.   Analogs  of (\ref{1})-(\ref{2}) as a function of $\overset{\leftharpoonup}{\ell}$ are presented  in Lemma \ref{thrm:spiked_covar} below. On this  scale, the analog of the BBP phase transition—--the critical spike strength above which leading sample eigenvectors correlate with population eigenvectors—--now occurs at $\overset{\leftharpoonup}{\ell} = 1$. These formulas and concepts are our basis for our development of new optimal eigenvalue shrinkers.  Results equivalent to Lemma \ref{thrm:spiked_covar} were previously established under slightly different assumptions by Bloemendal  et al.\ \cite{BKYY16}.  We provide a simple, direct argument that permits completely general rates at which $n, p \rightarrow \infty$ while $\gamma_n \rightarrow 0$. Analogous results hold as $\gamma_n \rightarrow \infty$, explored in later sections.

    \item  New sets of asymptotically optimal formulas for shrinkage of eigenvalues are found under fifteen canonical loss functions. While all losses  vanish as $\gamma_n \rightarrow 0$ and diverge as $\gamma_n \rightarrow \infty$, optimal shrinkage provides improvement by multiplicative factors. Under proportional growth, there is a distinct optimal shrinker for each of the fifteen losses considered. In the disproportionate regime, there are three distinct shrinkers, a simplification. These formulas are the limits of proportional shrinkers as the aspect ratio $\gamma_n$ vanishes or diverges and population spikes decay towards one or diverge. We derive closed forms for the relative gain of optimal shrinkage versus classical no-shrinkage. In addition, we find optimal hard thresholding levels under each loss.

    \item   The $n, p \rightarrow \infty$, $\gamma_n \rightarrow 0$ limit is dissimilar to classical fixed-$p$ statistics: for any rate $\gamma_n \rightarrow 0$, non-trivial eigenvalue shrinkage is optimal.

    \item A consequence of our results is that the optimal shrinkage formulas inherited from the proportional regime, with $\gamma$ replaced by $\gamma_n$, achieve optimal performance in the disproportionate limits. These shrinkers are harmless in the classical fixed-$p$ limit. A practitioner with a dataset of size $(n_{\text{data}}, p_{\text{data}})$ implementing the proportionate shrinkers of Donoho, Gavish, and Johnstone \cite{DGJ18} would substitute $p_{\text{data}}/n_{\text{data}}$ for $\gamma$. We prove that this heuristic is asymptotically optimal in the disproportionate limits. Thus, for a given loss function, we provide a single shrinkage function to be used with the current aspect ratio that achieves optimal performance in any asymptotic embedding of $(n_{\text{data}}, p_{\text{data}})$. A practitioner need not have the asymptotic embedding (unknown, of course, in practice) to perform optimal shrinkage.

    \item We obtain asymptotically optimal shrinkage formulas for the spiked Wigner model. As the empirical spectral distributions of the spiked covariance as $\gamma_n \rightarrow 0$  and the spiked Wigner converge to a common limit---the semicircle law---optimal shrinkage formulas are equivalent after a change of variables.
    \end{enumerate}

    Our assumption that non-spiked population eigenvalues are identity is for convenience. In the case of an arbitrary noise level, where $\Sigma$ is a low-rank perturbation of $\sigma^2 I$, procedures herein may be appropriately scaled. If the noise level $\sigma^2$ is unknown, it is consistently estimated by the median eigenvalue of $S$ as $\gamma_n  \rightarrow 0$. As $\gamma_n \rightarrow \infty$, the median of non-zero eigenvalues suffices. Asymptotic optimality extends to this setting by the continuity of the provided shrinkers. Knowledge of the number of spikes $r$ is similarly unnecessary. In practice, optimal shrinkers may be applied to each empirical eigenvalue. Rigorous proof of this claim is given in Section 7 of \cite{DGJ18}. Similarly, the rank and variance assumptions (\ref{model2}) may be relaxed.

\section{The Fixed-Spike, $\gamma_n \rightarrow \gamma$ Limit}
\label{sec-PGLim}

We briefly review certain tools and background concepts.

\begin{definition} As discussed in Section \ref{1.7}, the model rank $r$ is assumed known. It therefore makes sense to employ
 {\it rank-aware covariance shrinkage estimators}: for a shrinkage function
 $\eta: [0, \infty) \rightarrow [0, \infty)$,
 \begin{align}
 \widehat{\Sigma}_\eta   =  \widehat{\Sigma}_{\eta, n,r} & = \sum_{i=1}^r \eta(\lambda_i) v_i v_i' + \sum_{i=r+1}^n v_i v_i'    \nonumber \\
 & = \sum_{i=1}^r (\eta(\lambda_i) - 1) v_i v_i' + I \, .
 \label{000}
\end{align}
In the particular case $\eta(\lambda) \equiv \lambda$ ---no shrinkage---
we may instead write $S^r$ rather than $\widehat{\Sigma}_{\lambda, n,r}$.
\end{definition}

\begin{definition}
Let $\|\cdot\|_F$, $\|\cdot \|_{op}$, and $\|\cdot\|_*$ respectively
denote the Frobenius, operator, and nuclear matrix norms.
We consider estimation under 15 loss functions,
each formed by applying
one of the 3 matrix norms to one of 5 {\it pivots}.
By pivot, we mean a matrix-valued function
$\Delta(A,B)$ of two real positive definite matrices $A, B$;
we consider specifically:
\begin{align*}
 \Delta_1 &= A - B \,, & \Delta_2 &= A^{-1} - B^{-1}  \,, & \Delta_3 &= A^{-1} B - I   \,,
\\
 \Delta_4 &= B^{-1} A - I \,, & \Delta_5&= A^{-1/2} B A^{-1/2} - I  \, . &&
\end{align*}
We apply each norm to each of the pivots,
obtaining for $k=1, \ldots, 5$, the loss functions:
\begin{align}
   & L_{F,k}(\Sigma, \widehat \Sigma) = \|\Delta_k(\Sigma, \widehat \Sigma) \|_F \,,  \nonumber  & L_{O,k}(\Sigma, \widehat \Sigma) = \|\Delta_k(\Sigma, \widehat \Sigma) \|_{op} \,, \nonumber  && L_{N,k}(\Sigma, \widehat \Sigma) = \|\Delta_k(\Sigma, \widehat \Sigma) \|_* \, .
\end{align}
\end{definition}

\begin{lemma} \label{lem:dg18_7} (Lemma 7 of \cite{DGJ18})
In the proportional limit  (\ref{pro}) of the spiked model {\normalfont \bf I}, suppose
 $\eta(\lambda_i)$ has almost sure limit $\eta_i$,  $1 \leq i \leq r$.
Each loss $L_{\star,k}$ converges almost surely to a deterministic
limit:
\[
  L_{\star,k}(\Sigma, \widehat \Sigma_{\eta})   \xrightarrow{a.s.}
  {\cal L}_{\star,k}((\ell_i)_{i=1}^r,(\eta_i)_{i=1}^r) , \qquad \star \in \{ F,O,N\}, \quad 1 \leq k \leq 5 .
\]
The asymptotic loss is
sum- or max- decomposable
into $r$ terms deriving from non-unit spiked eigenvalues.
The terms involve matrix norms applied to pivots of
$2 \times 2$ matrix expressions:
\begin{align*}
   & A(\ell) = \begin{bmatrix} \ell & 0 \\ 0 & 1 \end{bmatrix} \,, & B(\eta,c) = I_2 + (\eta-1) \begin{bmatrix} c^2 & c s \\ c s & s^2 \end{bmatrix} \,,
\end{align*}
where $s^2 = 1 -c^2$. With $\ell_i$ denoting a spiked eigenvalue and  $c(\ell_i)$ the limiting cosine in (\ref{4}), the decompositions are
 \begin{align*} \phantom{\,,} {\cal L}_{F, k}((\ell_i)_{i=1}^r, (\eta_i)_{i=1}^r) & = \bigg(  \sum_{i=1}^r \big[ L_{F, k} \big(A(\ell_i),B(\eta_i,c(\ell_i)) \big) \big]^2 \bigg)^{1/2} \,, \\
 {\cal L}_{O,k}((\ell_i)_{i=1}^r, (\eta_i)_{i=1}^r) & = \max_{1 \leq i \leq r} L_{O,k} \big( A(\ell_i),B(\eta_i,c(\ell_i) )\big) \, , \\
 {\cal L}_{N,k}((\ell_i)_{i=1}^r,(\eta_i)_{i=1}^r) & = \sum_{i=1}^r L_{N,k} \big( A(\ell_i),B(\eta_i,c(\ell_i)) \big) \, .
\end{align*}
 \end{lemma}

For each of the 15 losses studied here, and several others, \cite{DGJ18} derives---under proportional growth $\gamma_n \rightarrow \gamma > 0$---a shrinker $\eta^*(\lambda | L) = \eta^*(\lambda| L, \gamma)$  minimizing the asymptotic loss  ${\cal L}$. Optimal shrinkers depend on the ``inverse" of the eigenvalue mapping (\ref{bias_func}):
\begin{align*} \ell(\lambda) = \ell(\lambda, \gamma) =
    \begin{dcases}
    \frac{\lambda +1 - \gamma + \sqrt{(\lambda - 1 - \gamma)^2 - 4 \gamma}}{2}  & \lambda >  \lambda_+(\gamma)  \\
     \ell_+(\gamma) & \lambda  \leq  \lambda_+(\gamma)
     \end{dcases} \, .
\end{align*}
Although $\ell_i$ is unobserved, this inverse provides a consistent estimator of $\ell_i$ above the BBP transition:
\[ \phantom{\,.} \ell(\lambda_i) \xrightarrow[]{a.s.} \ell_i \, , \hspace{2cm} \ell_i > \ell_+(\gamma)
\,.
\]

In most cases,  shrinkers are explicit in terms of $\ell$, $c$, and $s$. For example, for $L_{F,1}$, the optimal shrinker is $\eta^*(\lambda | L_{F,1}) = \ell(\lambda) \cdot c^2(\ell(\lambda)) + s^2(\ell(\lambda))$, while for $L_{O,1}$, it is simply $\eta^*(\lambda | L_{O,1}) = \ell(\lambda)$; a list of 18  such closed forms can be found in  \cite{DGJ18}. For notational lightness, we may write $\eta^*(\lambda | L_{F,1}) = \ell \cdot c^2 + s^2$, or $\eta^*(\lambda | L_{O,1}) = \ell$.

\section{Covariance Estimation as $\gamma_n \rightarrow 0$}
\subsection{The Variable-Spike, $\gamma_n \rightarrow 0$ Limit}
\label{sec-DPGLim}

We now formalize our earlier informal discussion of the asymptotic limit $\gamma_n \rightarrow 0$.
Consider the normalized empirical eigenvalues defined by
\begin{equation} \label{def-tlam}
 \tlam_i =  \tlam_{i,n} =  \frac{\lambda_i-1}{\sqrt{\gamma_n}}\,, \hspace{2cm} 1 \leq i \leq p \, .
\end{equation}
This transformation ``spreads" eigenvalues tightly grouped
near $1$. In the standard normal case $\Sigma = I$, the empirical distribution
of $(\tlam_i)_{i=1}^{p}$ converges weakly almost surely to the semicircle law with
support $[-2,2]$ (Bai and Yin \cite{BY88}).

We generalize the spiked model to allow spiked eigenvalues $(\ell_i)_{i=1}^r$ to vary with $n$. Matching (\ref{def-tlam}), we consider normalized spiked eigenvalues $\gamma_n^{-1/2}(\ell_i-1)$ and assume their convergence to limits $(\overset{\leftharpoonup}{\ell}_i)_{i=1}^r \in (0,\infty)$. That is, we study spiked eigenvalues of the form
\begin{align*}
 \hspace{-.1cm} [\textbf{II}]\hspace{1.5cm} \ell_{i} = \ell_{i, n} = 1 +  \overset{\leftharpoonup}{\ell}_i  (1+ o(1)) \sqrt{\gamma_n} \, , \hspace{2cm}  1 \leq i \leq r \, , \end{align*}
 where $(\overset{\leftharpoonup}{\ell}_i)_{i=1}^r$ are constant, non-negative, parameters.  Spiked eigenvalues are no longer fixed as under model {\bf I}, rather, they decay towards one at the $\sqrt{\gamma_n}$ rate. We assume supercritical eigenvalues---those with $\overset{\leftharpoonup}{\ell}_i > 1$---are simple.

This new disproportional limit and varying-spike model yields formulas analogous to, yet distinct from,  those in effect under proportional growth and fixed spikes. New formulas seem more elegant: a phase transition occurs simply at $\overset{\leftharpoonup}{\ell}=1$. Define the eigenvalue mapping function
\begin{equation} \label{def-dpg-spike-eigenmap}
  \tlam(\overset{\leftharpoonup}{\ell}) =  \begin{dcases}
                                 \overset{\leftharpoonup}{\ell} + \frac{1}{\overset{\leftharpoonup}{\ell}}  & \overset{\leftharpoonup}{\ell} > 1\\
                           2  & 0 < \overset{\leftharpoonup}{\ell} \leq 1 \\
                               \end{dcases}
\end{equation}
and the eigenvector cosine function
\begin{equation} \label{def-dpg-spike-cosine}
   \overset{\leftharpoonup}{c}^2(\overset{\leftharpoonup}{\ell})  =
             \ \begin{dcases}
                         1 - \frac{1}{\overset{\leftharpoonup}{\ell}^2} ,  &    \overset{\leftharpoonup}{\ell} > 1  \\
                         0  & 0 < \overset{\leftharpoonup}{\ell} \leq 1, \\
                                 \end{dcases}                                 .
\end{equation}
For future use, we also define $\overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell}) = 1-\overset{\leftharpoonup}{c}^2(\overset{\leftharpoonup}{\ell})$.
\begin{lemma} \label{thrm:spiked_covar}
Under $\gamma_n \rightarrow 0$ and varying spikes {\normalfont \bf II},
\begin{equation} \label{eq-dpg-spike-eigenmap}
\phantom{\,.} \tlam_{i}  \xrightarrow{a.s.} \tlam(\overset{\leftharpoonup}{\ell}_i) \, , \hspace{2cm} 1 \leq i \leq r \, .
\end{equation}
With $v_1, \ldots, v_{p}$ and denoting the eigenvectors of $S$ in decreasing eigenvalue ordering,
 and $u_1, \ldots, u_{p}$ the corresponding eigenvectors of $\Sigma$, the angles between pairs of eigenvectors have limits
\begin{equation} \label{lim-spg-spike-cosine}
 | \langle u_i, v_j \rangle | \xrightarrow{a.s.}  \delta_{ij} \cdot \overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell}_i), \hspace{1.05cm}  1 \leq i, j \leq r  \, .
\end{equation}
\end{lemma}

The reader will no doubt see that Lemma \ref{thrm:spiked_covar} exhibits a formal similarity to proportional regime results (\ref{1}) and (\ref{4}); as in the proportional case, spiked eigenvalues of a critical scale produce  eigenvalue inflation and eigenvector rotation, now written in terms of the new parameter $\overset{\leftharpoonup}{\ell}$. The arrow decorators allow us to preserve a formal resemblance between (\ref{bias_func}) and (\ref{2}) and previous  results, yet indicate there are important differences.

 A stronger, non-asymptotic  form of this lemma is established by Bloemendal et al.\ \cite{BKYY16}. Yet, \cite{BKYY16} requires the existence of some $k>0$ such that $n \leq p^k $ eventually, while here $\gamma_n$ may tend to zero arbitrarily slowly. Feldman \cite{F21} considers a closely-related signal-plus-noise model. For the reader's convenience, a simple, direct proof of Lemma \ref{thrm:spiked_covar} is provided in Appendix \ref{Appendix}.

\subsection{Asymptotic Loss in the Variable-Spike, $\gamma_n \rightarrow 0$ Limit}

Recall the families of rank-aware estimates $\widehat{\Sigma}_\eta$ and losses $L_{\star,k}$ defined in Section \ref{sec-PGLim}.  Under variable-spikes {\bf II},
the sequence of estimands
 \[
 \Sigma = \sum_{i=1}^r (\ell_i - 1) u_i u_i' + I
 \]
now approaches identity. $L_{\star, k}(\Sigma, \widehat \Sigma_\eta)$ vanishes for any shrinker $\eta$ that is continuous at one and satisfies $\eta(1) = 1$, in particular, $L_{\star,k}(\Sigma, S) \rightarrow 0$. We therefore consider rescaled losses:
\[
\lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{\star, k}(\Sigma,\widehat \Sigma) =  \frac{L_{\star,k}(\Sigma, \widehat \Sigma)}{\sqrt{\gamma_n}} \, .
\]
Observe that
$\lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{\star, 1}(\Sigma, \widehat \Sigma)  = \|(\Sigma - I) - (\widehat{\Sigma} - I) \|_{\star} /\sqrt{\gamma_n} $,
which we view as transforming to a new coordinate system with origin at the identity matrix. Let $\lharp \phi_n(x) = \gamma_n^{-1/2}(x-1)$ denote the mapping to these coordinates. Thus, (\ref{def-tlam}) and spikes {\bf II} may be written as $\tlam_{i} = \lharpoonu{\phi}_n(\lambda_{i})$
and $\lharpoonu{\phi}_n(\ell_{i}) \rightarrow \overset{\leftharpoonup}{\ell}_i$.

\begin{definition}
Let $\eta = \eta_n$ denote a sequence of shrinkers, possibly varying with $n$. Suppose that under the disproportional $\gamma_n \rightarrow 0$ limit and varying-spikes {\bf II}, the sequences of normalized shrinker outputs induced by rescaling converge:
\[   \phantom{\,.}   \lharpoonu{\phi}_n(\eta(\lambda_i)) \xrightarrow{a.s.} \lharp \eta_i     \, , \hspace{2cm} 1 \leq i \leq r \, .  \]
We call the limits $(\lharp \eta_i)_{i=1}^r$ the {\it asymptotic shrinkage descriptors}.
\end{definition}

\begin{lemma} \label{lem-dpg-asy-loss} Let $\eta$ denote a sequence of shrinkers with asymptotic shrinkage descriptors $(\lharp \eta_i)_{i=1}^r$ under the disproportional $\gamma_n \rightarrow 0$ limit and varying-spikes {\normalfont \bf II}. Each loss $\lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{\star,k}$ then converges almost surely to a deterministic limit:
\[
  \lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{\star,k}(\Sigma, \widehat{\Sigma}_{\eta})  \xrightarrow{a.s.}
  \lharpoonu{\cal L}_{\star}((\overset{\leftharpoonup}{\ell}_i)_{i=1}^r, (\overset{\leftharpoonup}{\eta}_i)_{i=1}^r) \, , \qquad \star \in \{ F,O,N\} \, , \quad 1 \leq k \leq 5 \, .
\]
The asymptotic loss does not involve $k$. It is  sum- or max- decomposable into $r$ terms deriving from non-unit spike eigenvalues. The terms involve matrix norms applied to pivots of $2 \times 2$ matrices $\widetilde{A}$ and $\widetilde{B}$:
\begin{align*}
   & \widetilde{A}(\overset{\leftharpoonup}{\ell}) = \begin{bmatrix} \overset{\leftharpoonup}{\ell} & 0 \\ 0 & 0 \end{bmatrix} \,, & \widetilde{B}(\overset{\leftharpoonup}{\eta},\overset{\leftharpoonup}{c}) =  \overset{\leftharpoonup}{\eta} \cdot \begin{bmatrix} \overset{\leftharpoonup}{c}^2 & \overset{\leftharpoonup}{c} \overset{\leftharpoonup}{s} \\ \overset{\leftharpoonup}{c} \overset{\leftharpoonup}{s} & \overset{\leftharpoonup}{s}^2 \end{bmatrix} \,,
\end{align*}
where $\lharp s \mystrut^2 = 1 - \lharp c \mystrut^2$.  With $\lharp \ell_i$ denoting a spiked eigenvalue and $\lharp c(\lharp \ell_i)$ the limiting cosine in (\ref{lim-spg-spike-cosine}), the decompositions are
 \begin{align*} \phantom{\,,} \lharpoonu{\cal L}_{F}((\overset{\leftharpoonup}{\ell}_i)_{i=1}^r, (\overset{\leftharpoonup}{\eta}_i)_{i=1}^r) & = \bigg( \sum_{i=1}^r \big[ L_{F, 1}\big(\widetilde{A}(\overset{\leftharpoonup}{\ell}_i),\widetilde{B}(\overset{\leftharpoonup}{\eta}_i,\overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell}_i)) \big) \big]^2 \bigg)^{1/2} \,, \\
 \lharpoonu{\cal L}_{O}((\overset{\leftharpoonup}{\ell}_i)_{i=1}^r, (\overset{\leftharpoonup}{\eta}_i)_{i=1}^r) & = \max_{1 \leq i \leq r} L_{O,1} \big(\widetilde{A}(\overset{\leftharpoonup}{\ell}_i),\widetilde{B}(\overset{\leftharpoonup}{\eta}_i,\overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell}_i))\big) \, , \\
 \lharpoonu{\cal L}_{N}((\overset{\leftharpoonup}{\ell}_i)_{i=1}^r,(\overset{\leftharpoonup}{\eta}_i)_{i=1}^r) & = \sum_{i=1}^r L_{N,1} \big(\widetilde{A}(\overset{\leftharpoonup}{\ell}_i),\widetilde{B}(\overset{\leftharpoonup}{\eta}_i,\overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell}_i)) \big) \, .
\end{align*}
 \end{lemma}
\begin{proof}
Under loss $L_{\star,1}$, the argument is identical to the proofs of Lemma 2 and 7 of \cite{DGJ18}, only using Lemma \ref{thrm:spiked_covar} for eigenvalue inflation and eigenvector rotation as $\gamma_n \rightarrow 0$. The pivots we consider are asymptotically equivalent: using the simultaneous block decomposition in Lemma 5 of \cite{DGJ18} and a Neumann series expansion,
\[
\phantom{\,. }
|\lharp L_{\star,1}(\Sigma, \widehat{\Sigma}_{\eta}) - \lharp L_{\star, k}(\Sigma, \widehat{\Sigma}_{\eta})| \xrightarrow[]{a.s.} 0 \, , \hspace{2cm} 2 \leq k \leq 5 \, .   \]
\end{proof}
For example, the asymptotic shrinkage descriptors of the  rank-aware sample covariance estimator $S^r = S_n^r = \sum_{i=1}^r (\lambda_i - 1) v_i v_i' + I$ are $\lharp \eta_i = \lharp \lambda(\lharp \ell_i)$. For $r = 1$, suppressing the subscript of $\overset{\leftharpoonup}{\ell}_1$, squared asymptotic loss evaluates to
\begin{equation} \label{eq-dpg-shr-asy-loss-fro}
\phantom{\,.}  \big[\lharpoonu{\cal L}_{F}(\overset{\leftharpoonup}{\ell}, \tlam(\overset{\leftharpoonup}{\ell}))\big]^2 =  (\overset{\leftharpoonup}{\ell} - \tlam(\overset{\leftharpoonup}{\ell}) \overset{\leftharpoonup}{c}^2(\overset{\leftharpoonup}{\ell}))^2 + \tlam^2(\overset{\leftharpoonup}{\ell}) (1-\overset{\leftharpoonup}{c}^4(\overset{\leftharpoonup}{\ell}))  \,  .
\end{equation}
By Lemma \ref{thrm:spiked_covar}, this simplifies to $2+ 3/\overset{\leftharpoonup}{\ell}^2$ for $\overset{\leftharpoonup}{\ell} > 1$ and to $\overset{\leftharpoonup}{\ell}^2+4$ for $\overset{\leftharpoonup}{\ell} \leq 1$. Hence, the (unsquared) asymptotic loss attains a global maximum of $\sqrt{5}$ precisely at the phase transition $\overset{\leftharpoonup}{\ell}=1$. Asymptotic losses of $S^1$ under each norm are collected below in Table 1, to later facilitate comparison with optimal shrinkage.

\setlength\extrarowheight{5pt}
 \begin{table}[h]
 \centering
 \begin{tabular}{| c | c | c |}
\hline
 Norm & $\overset{\leftharpoonup}{\ell} < 1$ & $\overset{\leftharpoonup}{\ell} > 1$ \\
\hline
Frobenius & $\sqrt{\lharp \ell^2 + 4}$  & $\sqrt{2 + 3/\overset{\leftharpoonup}{\ell}^2}$ \\
Operator & $2$ & $ \big ( 1+ \sqrt{5+ 4 \overset{\leftharpoonup}{\ell}^2} \big) /(2\overset{\leftharpoonup}{\ell})$ \\
Nuclear & $\overset{\leftharpoonup}{\ell} + 2 $ & $\sqrt{4+5/\overset{\leftharpoonup}{\ell}^2}$ \\
\hline
 \end{tabular}
 \caption{Asymptotic Loss $\lharpoonu{\cal L}_\star$ of the rank-aware sample covariance $S^1$ (the subscript of $\overset{\leftharpoonup}{\ell}_1$ is suppressed).}
 \label{tbl-asy-loss-shr-rank-aware}
 \end{table}
\setlength\extrarowheight{0pt}

\subsection{Optimal Asymptotic Loss}

This subsection assumes $r=1$; the subscript of $\overset{\leftharpoonup}{\ell}_1$ is suppressed. Recalling the relations between $\overset{\leftharpoonup}{\ell}$, $\tlam(\overset{\leftharpoonup}{\ell})$, and $\overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell})$,
one sees in Lemma \ref{lem-dpg-asy-loss} and (\ref{eq-dpg-shr-asy-loss-fro}) that $\tlam(\overset{\leftharpoonup}{\ell})$ is not the minimizer of the function $\overset{\leftharpoonup}{\eta} \mapsto \lharpoonu{\cal L}_{\star}(\overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta})$. A sequence estimators $\widehat \Sigma_{\eta} = (\eta(\lambda_1)-1) v_1 v'_1 + I $ can outperform $S^1$ substantially, provided the  asymptotic shrinkage descriptor $\overset{\leftharpoonup}{\eta}_1 = \lim_{n \rightarrow \infty} \lharp \phi_n(\eta(\lambda_1)) $ exists and $\lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta}_1) < \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell}, \tlam(\overset{\leftharpoonup}{\ell}))$.

In this subsection, we calculate the asymptotic shrinkage descriptors that minimize $\overset{\leftharpoonup}{\eta} \mapsto \lharpoonu{\cal L}_{\star}(\overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta})$. The following subsection shows the existence of shrinkers with such asymptotic shrinkage descriptors.

\begin{definition}
\label{def-shr-formal-asy}
The  {\it formally optimal asymptotic loss} in the rank-1 setting is
\[
\lharpoonu{\cal L}_{\star}^1(\overset{\leftharpoonup}{\ell})  = \min_{\vartheta} \lharpoonu{\cal L}_{\star}(\overset{\leftharpoonup}{\ell},\vartheta), \qquad \star \in \{F,O,N\}.
\]
A  { \it formally optimal shrinker} is a function $\overset{\leftharpoonup}{\eta}(\cdot | \star): \mathbb{R} \mapsto \mathbb{R}$
achieving $\lharpoonu{\cal L}_{\star}^1(\overset{\leftharpoonup}{\ell})$:
\[
     \overset{\leftharpoonup}{\eta}(\overset{\leftharpoonup}{\ell} | \star) = \underset{\vartheta}{\mbox{argmin}} \, \lharpoonu{\cal L}_{\star}(\overset{\leftharpoonup}{\ell},\vartheta), \qquad \overset{\leftharpoonup}{\ell} > 0 \, ,  \qquad \star \in \{F,O,N\} \, .
\]
We write $\lharp \eta(\overset{\leftharpoonup}{\ell} | \star)$ rather than $\lharp \eta(\overset{\leftharpoonup}{\ell} | L_{\star,k})$ as by Lemma \ref{lem-dpg-asy-loss}, optimal asymptotic losses are independent of the pivot $k$.
\end{definition}

\begin{lemma}
\label{lem-shr-opt}
Formally optimal shrinkers and corresponding losses are given by
\setlength\extrarowheight{3pt}
\begin{align}
    & \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell}|F) = (\overset{\leftharpoonup}{\ell} - 1/\overset{\leftharpoonup}{\ell})_+ \,,&&   \lharpoonu{\cal L}_{F}^1(\overset{\leftharpoonup}{\ell}) =
   \begin{dcases}
               \sqrt{2 - 1/\overset{\leftharpoonup}{\ell}^2} & \overset{\leftharpoonup}{\ell} > 1 \\
               \overset{\leftharpoonup}{\ell} & 0 < \overset{\leftharpoonup}{\ell} \leq 1
               \end{dcases}
         \, , \nonumber  \\
    & \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell}|O) =
    \overset{\leftharpoonup}{\ell} \cdot 1_{\{\overset{\leftharpoonup}{\ell} > 1\}}
               \,, &&\lharpoonu{\cal L}_{O}^1(\overset{\leftharpoonup}{\ell}) = \begin{dcases}
               1 & \overset{\leftharpoonup}{\ell} > 1\\
               \overset{\leftharpoonup}{\ell} & 0 < \overset{\leftharpoonup}{\ell} \leq 1
               \end{dcases} \, , \label{003}\\
    & \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell}|N) = \big(\overset{\leftharpoonup}{\ell} -2 /\overset{\leftharpoonup}{\ell}\big)_+ \, , && \lharpoonu{\cal L}_{N}^1(\overset{\leftharpoonup}{\ell}) =
     \begin{dcases}
               2\sqrt{1-1/\overset{\leftharpoonup}{\ell}^2} & \overset{\leftharpoonup}{\ell} > \sqrt{2} \\
               \overset{\leftharpoonup}{\ell} & 0 < \overset{\leftharpoonup}{\ell} \leq \sqrt{2}
               \end{dcases}
\,  . \nonumber
    \end{align}
\setlength\extrarowheight{0pt}
\end{lemma}
\begin{proof}
By Lemma \ref{lem-dpg-asy-loss},
\begin{align} \label{1523} \phantom{\,.} \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell}, \vartheta) & = L_{\star,1}\big(\widetilde A(\overset{\leftharpoonup}{\ell}), \widetilde B(\vartheta, \lharp c(\overset{\leftharpoonup}{\ell})) \big) = \bigg \|  \begin{bmatrix} \overset{\leftharpoonup}{\ell} - \vartheta \lharp c \mystrut^2(\overset{\leftharpoonup}{\ell}) & - \vartheta \lharp c(\overset{\leftharpoonup}{\ell}) \lharp s(\overset{\leftharpoonup}{\ell}) \\ - \vartheta \lharp c(\overset{\leftharpoonup}{\ell}) \lharp s(\overset{\leftharpoonup}{\ell}) & -\vartheta \lharp s \mystrut^2(\overset{\leftharpoonup}{\ell}) \end{bmatrix}
\bigg\|_\star \nonumber \\
& =\begin{cases}
\sqrt{(\overset{\leftharpoonup}{\ell} - \vartheta)^2  + 2 \overset{\leftharpoonup}{\ell} \vartheta \overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell})} & \star = F \\
      \max(|\lambda_+|, |\lambda_-|) & \star = O \\
      |\lambda_+| + |\lambda_-| & \star = N
   \end{cases}
\,,\end{align}
where $\lambda_\pm = \big(\vartheta - \overset{\leftharpoonup}{\ell} \pm \sqrt{(\vartheta - \overset{\leftharpoonup}{\ell})^2 + 4 \vartheta \overset{\leftharpoonup}{\ell} \overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell})} \big)/2$ ($\lambda_\pm$ are the eigenvalues of the above $2 \times 2$ matrix, according to Lemma 14 of \cite{DGJ18s}).
Differentiating, Frobenius loss is minimized by $\vartheta = \overset{\leftharpoonup}{\ell} \lharp c \mystrut^2(\overset{\leftharpoonup}{\ell}) = (\overset{\leftharpoonup}{\ell} - 1/\overset{\leftharpoonup}{\ell})_+$.
For $\overset{\leftharpoonup}{\ell} > 1$, operator norm loss is minimized by $\vartheta = \overset{\leftharpoonup}{\ell}$, for which $\lambda_+ = - \lambda_-$. For $\overset{\leftharpoonup}{\ell} \leq 1$, $\lambda_+ = \vartheta$, while $-\lambda_- = \overset{\leftharpoonup}{\ell}$. In this case, we take $\vartheta = 0$. For $\vartheta \geq 0$, nuclear norm loss may be rewritten as
\[ \ \lharpoonu{\cal L}_N(\overset{\leftharpoonup}{\ell},\vartheta) =  \sqrt{(\vartheta - \overset{\leftharpoonup}{\ell})^2 + 4 \vartheta \overset{\leftharpoonup}{\ell} \overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell}) }  \, ;
\]
this is minimized by $\vartheta =  \overset{\leftharpoonup}{\ell} (1-2 \overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell}))_+$.
If $\theta \leq 0$, $\lharpoonu{\cal L}_N(\overset{\leftharpoonup}{\ell},\vartheta) = - \vartheta + \overset{\leftharpoonup}{\ell}$ is minimized by $\vartheta = 0$.
We collect below formally optimal shrinkers:
\begin{align} \label{9237}
 &    \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell} | F) = \overset{\leftharpoonup}{\ell} \overset{\leftharpoonup}{c}(\overset{\leftharpoonup}{\ell}) \, , &   \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell} | O) = \overset{\leftharpoonup}{\ell} \cdot 1_{\{\overset{\leftharpoonup}{\ell} > 1\}} \, , &&   \overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell} | N) = \overset{\leftharpoonup}{\ell} (1-2 \overset{\leftharpoonup}{s}^2(\overset{\leftharpoonup}{\ell}))_+ \, .
\end{align}
Substitution of (\ref{def-dpg-spike-cosine}) completes the proof.
\end{proof}

\subsection{Unique Admissibility}

Formally optimal shrinkers derived in the previous subsection depend on $\overset{\leftharpoonup}{\ell}$, which is not observable. We define the partial inverse of the eigenvalue mapping $\tlam(\overset{\leftharpoonup}{\ell})$ (\ref{def-dpg-spike-eigenmap}):

\begin{equation} \label{def-invert-eigenmap}
  \overset{\leftharpoonup}{\ell}( \tlam)  =
  \left \{ \begin{array}{ll}
  (\tlam + \sqrt{\tlam^2 - 4})/2 \ &  \tlam > 2 \\
  1 & \tlam \leq 2 \\
    \end{array}
   \right .  .
 \end{equation}
Recall the rescaling mapping $\lharpoonu{\phi}_n$, with inverse $\lharpoonu{\phi}_n^{-1}(\overset{\leftharpoonup}{\eta}) = 1 + \sqrt{\gamma_n} \overset{\leftharpoonup}{\eta}$. Through these functions, we may ``change coordinates'' in shrinkers defined in terms of $\overset{\leftharpoonup}{\ell}$ to obtain shrinkers defined on observables.
Thanks to the sum-/max-
decomposibility of asymptotic losses, these shrinkers generate covariance estimates which are asymptotically optimal in the rank-$r$ case.
\begin{definition} \label{def48}
    A sequence of shrinkers $\eta^*(\lambda|\star) = \eta_n^*(\lambda|\star)$ is {\it asymptotically optimal} under the disproportional limit $\gamma_n \rightarrow 0$, variable spikes  {\bf II}, and loss $\lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{\star,k}$ if the formally optimal asymptotic loss is achieved:
\begin{align*}
\phantom{\,.} \lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{F,k}(\Sigma,\widehat{\Sigma}_{\eta^*(\lambda|F)})  & \xrightarrow{a.s.} \bigg( \sum_{i=1}^r \big[ \lharpoonu{\cal L}_F^1(\overset{\leftharpoonup}{\ell}_i) \big]^2 \bigg)^{1/2} \, ,  \\
\phantom{\,.} \lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{O,k}(\Sigma,\widehat{\Sigma}_{\eta^*(\lambda|O)})  & \xrightarrow{a.s.} \max_{1 \leq i \leq r}  \lharpoonu{\cal L}_O^1(\overset{\leftharpoonup}{\ell}_i) \, , \\
\phantom{\,.} \lharp L} %\newcommand{\tEll}{\lharpoonu{L}_{N,k}(\Sigma,\widehat{\Sigma}_{\eta^*(\lambda|N)})  & \xrightarrow{a.s.}  \sum_{i=1}^r \lharpoonu{\cal L}_N^1(\overset{\leftharpoonup}{\ell}_i)  \, .
\end{align*}
\end{definition}

\begin{theorem} \label{thrm:spiked_covar2}
For $\star \in \{F,N\}$, define the following shrinkers through the formally optimal shrinkers $\overset{\leftharpoonup}{\eta}(\overset{\leftharpoonup}{\ell} | \star)$ of Lemma \ref{lem-shr-opt}:
\begin{align}
   \eta^*(\lambda | \star)  & =   \lharpoonu{\phi}_n^{-1} \big (\overset{\leftharpoonup}{\eta}^*(\overset{\leftharpoonup}{\ell}(\lharpoonu{\phi}_n(\lambda))|\star )\big) \label{def-asy-opt-shrink} \nonumber \\
          & = 1 + \sqrt{\gamma_n} \cdot \overset{\leftharpoonup}{\eta}^* \Big( \overset{\leftharpoonup}{\ell} \Big( \frac{\lambda-1}{\sqrt{\gamma_n}} \Big) \Big| \star \Big) \, .
\end{align}
For the operator norm,  define

\begin{align} \eta^*(\lambda|O)  &= \lharp \phi_n^{-1} \Big(\lharp \ell(\lharp \phi_n(\lambda))  \cdot 1_{\{ \lharp \phi_n(\lambda) > \tau_n  \}} \Big) \nonumber \\
&= 1 + \sqrt{\gamma_n} \cdot \lharp \ell  \Big( \frac{\lambda-1}{\sqrt{\gamma_n}} \Big)   \cdot 1_{\{ \lambda > 1+ \tau_n \sqrt{\gamma_n}  \}} \,, \label{7059}
\end{align}
where $\tau_n = 2 + o(1)$ is a sufficiently slowly decaying sequence.

The shrinkers $\eta^*(\cdot|\star)$ are asymptotically optimal. Consider any other sequence of shrinkers $\eta^\circ = \eta_n^\circ$; unless $\eta^\circ$ has asymptotic shrinkage descriptors equal to those of $\eta^*$, it has strictly larger asymptotic normalized loss. Thus, up to asymptotically negligible perturbations, $\eta^*(\cdot | \star)$ is the unique asymptotically admissible shrinker.
\end{theorem}

Empirically, for the operator norm, bulk edge thresholding performs well:
\[ \phantom{\,.} \eta^*(\lambda | O) = \lharp \phi_n^{-1} \big(\lharp \ell(\lharp \phi_n(\lambda))  \cdot 1_{\{ \lharp \phi_n(\lambda) > 2 \}} \big) \,.
\]
This shrinker, which thresholds normalized eigenvalues exactly at two, is used in the simulations visualized in Figure \ref{fig-shr-empirical}. Achieved loss is quite close to $\lharp {\cal L}_O^1(\overset{\leftharpoonup}{\ell})$ on $(0,1]$. The slightly elevated threshold in (\ref{7059}) is an artifact of the proof.

\begin{proof}
By Lemma \ref{thrm:spiked_covar} and continuity of the partial inverse (\ref{def-invert-eigenmap}),
\begin{equation}
\phantom{\,.} \lharp \ell(\lharp \lambda_i) \xrightarrow{a.s.} \max(\lharp \ell_i, 1)  \, , \hspace{2cm}  1 \leq i \leq r \, . \label{0575}
\end{equation}
As $\lharp \eta^*(\overset{\leftharpoonup}{\ell}|F)$ and $\lharp \eta(\overset{\leftharpoonup}{\ell}|N)$ are continuous and on $(0,1]$ constant, (\ref{0575}) implies the asymptotic shrinkage descriptors of $\eta^*(\cdot|F)$ and $\eta^*(\cdot|N)$ almost surely exist and equal the formally optimal evaluates $(\lharp \eta^*(\overset{\leftharpoonup}{\ell}_i|F))_{i=1}^r$ and  $(\lharp \eta^*(\overset{\leftharpoonup}{\ell}_i|N))_{i=1}^r$, respectively. The formally optimal shrinker under operator norm loss $\lharp \eta^*(\cdot|O)$ is discontinuous at the phase transition $\overset{\leftharpoonup}{\ell} = 1$. For $\lharp \ell_i > 1$, existence and matching of the $i$-th asymptotic shrinkage descriptor to $\lharp \eta^*(\overset{\leftharpoonup}{\ell}_i|O)$ is immediate. Subcritical spiked eigenvalues converge to the bulk upper edge, at a rate bounded by equation (1.6) of \cite{F21}: for $\overset{\leftharpoonup}{\ell}_i \leq 1$, almost surely eventually, $\tlam_i \leq p^{-1/11}$.  The $i$-th asymptotic shrinkage descriptor is therefore zero.

Consider any other sequence of shrinkers $\eta^\circ = \eta^\circ_n$ and
rank-aware shrinkage estimators  $\widehat{\Sigma}_{\eta^\circ}$.  Unless this sequence has
asymptotic shrinkage descriptors
$(\overset{\leftharpoonup}{\eta}_i^\circ)_{i=1}^r$ identical to those of $\eta^*$, its asymptotic loss is strictly larger. Namely, if there is a subsequence with
normalized limits $\overset{\leftharpoonup}{\eta}_i^\circ = \lim_{k \rightarrow \infty} \lharpoonu{\phi}_{n_k}(\eta^\circ(\lambda_{i,n_k}))$
and $\| (\overset{\leftharpoonup}{\eta}_i^\circ)_{i=1}^r - (\overset{\leftharpoonup}{\eta}_i^*)_{i=1}^r \|_{\ell_{2}^r}  = \varepsilon > 0$, then by Lemmas \ref{lem-dpg-asy-loss} and \ref{lem-shr-opt}, there is some $\delta = \delta(\varepsilon) > 0$ such that along this subsequence, the asymptotic loss under shrinker $\eta^\circ$ exceeds the asymptotic loss of $\eta^*$ by at least $\delta$.

Note that $\eta^*$ does not depend on the model parameters $(\overset{\leftharpoonup}{\ell}_i)_{i=1}^r$. Hence,  $\widehat{\Sigma}_{\eta^*}$ achieves optimal asymptotic performance at each possible choice of $(\overset{\leftharpoonup}{\ell}_i)_{i=1}^r$. Except for asymptotically negligible perturbations, $\eta^*(\cdot|\star)$ is the unique shrinker asymptotically admissible under $L_{\star,k}$.
\end{proof}

\begin{corollary} \label{cor1} Under the disproportional limit $\gamma_n \rightarrow 0$ and variable spikes  {\bf II}, both the sample covariance $S$ and the rank-aware sample covariance
\begin{align} S^r = \sum_{i=1}^r (\lambda_i-1) v_i v_i' + I \label{11}
\end{align}
are asymptotically inadmissible for $L_{\star, k}$.
\end{corollary}

 \begin{proof}
This is an immediate consequence of Theorem \ref{thrm:spiked_covar2}.
Still, we sketch a direct argument for the Frobenius-norm case.
Let $W$ denote projection onto the combined span of $(u_i)_{i=1}^r$
and $(v_i)_{i=1}^r$. Then, for the sample covariance matrix,
\begin{eqnarray*}
\phantom{\,.} \| \Sigma - S \|_F^2 &=&  \| W'( \Sigma - S )W \|_F^2  +  \| (I-W)( I - S )(I-W) \|_F^2 \, ,
\end{eqnarray*}
while for the rank-aware estimate,
\begin{eqnarray*}
\phantom{\,.} \| \Sigma - S^r \|_F^2 &=&  \| W( \Sigma - S^r )W \|_F^2  +  \| (I-W)( I - S^r )(I-W) \|_F^2 \\
&=&  \| W( \Sigma - S^r )W \|_F^2  \, .
\end{eqnarray*}
The terms
 $\| W'( \Sigma - S )W \|_F^2$ and $ \| W( \Sigma - S^r )W \|_F^2$ tend to a common limit, so it suffices to study the rank-aware case. By Lemma \ref{lem-dpg-asy-loss}, $\|\Sigma - S^r\|_F^2 \xrightarrow{a.s.} \sum_{i=1}^r [\lharp {\cal L}(\overset{\leftharpoonup}{\ell}_i, \tlam(\overset{\leftharpoonup}{\ell}_i))]^2$,  and using equation (\ref{eq-dpg-shr-asy-loss-fro}) one may verify that
\[
\phantom{\,.} [\lharpoonu{\cal L}_{F}(\overset{\leftharpoonup}{\ell}, \tlam(\overset{\leftharpoonup}{\ell}))]^2 -   [\lharpoonu{\cal L}_{F}(\overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta}^*)]^2  = (\tlam(\overset{\leftharpoonup}{\ell}) - \overset{\leftharpoonup}{\eta}^*)^2 \geq 0 \, .
\]
Over the range $\overset{\leftharpoonup}{\ell} > 1$,  $\tlam(\overset{\leftharpoonup}{\ell}) - \overset{\leftharpoonup}{\eta}^* = 2/\overset{\leftharpoonup}{\ell}$, while over $\overset{\leftharpoonup}{\ell} \leq 1$, $\tlam(\overset{\leftharpoonup}{\ell}) - \overset{\leftharpoonup}{\eta}^* = 2$; thus the inequality is strict.
 \end{proof}

\subsection{Performance in the $\gamma_n \rightarrow 0$ Limit}

Figure \ref{fig-shr-optshrink} depicts optimal shrinkers (left) and
corresponding asymptotic losses (right, in the rank-one case $r=1$).  In the left-hand panel, the red curve marks the diagonal $\overset{\leftharpoonup}{\eta}(\tlam) = \tlam$,
corresponding to no shrinkage. For each loss function we consider, the
optimal shrinker $\overset{\leftharpoonup}{\eta}^*(\cdot| \star)$ lies below the diagonal. All optimal shrinkers vanish below the bulk edge, $\tlam \leq 2$.

Below the phase transition occuring at $\overset{\leftharpoonup}{\ell} = 1$, sample and population eigenvectors
are asymptotically orthogonal. In that region, it is futile to use empirical eigenvectors to model low-rank structure---they are pure noise. Therefore, to achieve optimal loss, we simply take $\eta=0$. According to
 (\ref{def-dpg-spike-eigenmap}), $\overset{\leftharpoonup}{\ell} \leq 1$ if and only if $\tlam \leq 2$,
hence all optimal rules vanish for $\tlam <  2$. Over the restricted range $0 < \overset{\leftharpoonup}{\ell} < 1$, optimal rules are of course not unique; we also  obtain  optimality over that range
by simple bulk-edge hard thresholding of empirical eigenvalues,
$\overset{\leftharpoonup}{\eta}(\tlam) = \tlam \cdot 1_{\{\tlam> 2\}}$.)

The right-hand panel compares performances under various loss functions of the standard estimator $S^r$ (dotted lines) and optimal estimators (solid lines). Asymptotic losses of the standard estimator are strictly larger than that of optimal estimators for all $\overset{\leftharpoonup}{\ell}$---near $\overset{\leftharpoonup}{\ell} = 1$, standard loss is far larger. As $\overset{\leftharpoonup}{\ell} \rightarrow 0^+$, optimal losses tend to zero, while standard losses tend to $2$.

\begin{definition}
The (absolute) {\it  regret} of a decision rule $\overset{\leftharpoonup}{\eta}$ is defined as
\[ \phantom{\,.} \lharp {\cal R}_\star(\overset{\leftharpoonup}{\ell},\overset{\leftharpoonup}{\eta}) = \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\overset{\leftharpoonup}{\eta}) - \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta}^*) \, .
\]
The {\it possible improvement} of a decision rule $\overset{\leftharpoonup}{\eta}$ is
$\lharp {\cal I}_\star(\overset{\leftharpoonup}{\ell},\overset{\leftharpoonup}{\eta}) = \lharp {\cal R}_\star(\overset{\leftharpoonup}{\ell},\overset{\leftharpoonup}{\eta})/\lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\overset{\leftharpoonup}{\eta})$,
i.e., the fractional amount by which performance improves by switching to the optimal rule.
\end{definition}

Losses of $S^r$ in the right-hand panel of Figure \ref{fig-shr-optshrink}
are well above losses of optimal estimators below the phase transition $\overset{\leftharpoonup}{\ell} \leq 1$;
the limit $\overset{\leftharpoonup}{\ell} \rightarrow 0^+$ produces maximal absolute regret,  $2$, for each of these losses.
For example, with operator norm loss, $\lharpoonu{\cal L}_O(0^+,\tlam) = 2$, while $\lharpoonu{\cal L}_O(0^+,\overset{\leftharpoonup}{\eta}^*)=0$,
giving absolute regret $\lharp {\cal R}_O(0^+,\tlam) =  2$ and possible improvement
$\lharp {\cal I}_O(0^+,\tlam) = 1$ (100\% of the standard loss is avoidable).
Similarly, with nuclear norm loss, we have $\lharp {\cal R}_N(\overset{\leftharpoonup}{\ell},\tlam)=2$ for $\overset{\leftharpoonup}{\ell} \leq 1$,
but $\lharp {\cal I}_N(0,\tlam) = 1$ (100\% of the standard loss is avoidable).
\setlength\extrarowheight{5pt}
\begin{table}
\centering
\begin{tabular}{| c | c | c | c | c |}
\hline
Norm & $\lharp {\cal R}_\star(0^+,\tlam)$ & $\lharp {\cal R}_\star(1,\tlam)$ & $\lharp {\cal I}_\star(0^+,\tlam)$ & $\lharp {\cal I}_\star(1,\tlam)$\\
\hline
Frobenius & $2$ & $\sqrt{5}-1$ & $100$\% & 55\%  \\
Operator &  2 &  1& $100$\% & 50\%  \\
Nuclear &  2 &   2& $100$\% & 66\%  \\
\hline
\end{tabular}
\caption{{\bf Regret and Improvement, $\gamma_n \rightarrow 0$.}
Absolute Regret $\lharp {\cal R}$ and Possible Improvement $\lharp {\cal I}$
of the standard rank-aware estimator $S^r$ (equivalently, $\overset{\leftharpoonup}{\eta} = \tlam$) near zero and exactly at the phase transition $\overset{\leftharpoonup}{\ell}=1$.}
\end{table}
\begin{figure}[h]
\centering
\includegraphics[height=2.6in]{OptNonlinGammaZero}
\includegraphics[height=2.6in]{spiked_covar2.png}
\caption{{\bf Optimal shrinkers and losses, $\gamma_n \rightarrow 0$.}
Left: optimal shrinkage functions. Right: losses of optimal shrinkers (solid) and of the standard estimator $S^r$ under Frobenius (blue), operator (orange), nuclear (green) norms.}
\label{fig-shr-optshrink}
\end{figure}
\setlength\extrarowheight{0pt}
\begin{figure}[h!]
\centering
\includegraphics[height=3in]{spiked_covar7.png}
\caption{{\bf Monte-Carlo simulations, small $\gamma_n$}.
Average over 50 realizations of losses under three norms, both
for the standard and asymptotically optimal estimators. Here, $p=1{,}000$ and $n = 100{,}000$, so $\gamma_n= .01$.}
\label{fig-shr-empirical}
\end{figure}

\section{Covariance Estimation as $\gamma_n \rightarrow \infty$}
\subsection{The Variable-Spike, $\gamma_n \rightarrow \infty$ Limit}
\label{sec-DPGLim-Infty}

\label{sec:spiked_covar2}

We now turn to the dual situation, $\gamma_n \rightarrow \infty$. To expose phase transition phenomena, we consider variable spiked eigenvalues of the form
\begin{align*}
 \hspace{-.1cm} [\textbf{III}]\hspace{1.5cm}    \ell_i = \ell_{i;n} = 1 + (\rharpoonu{\ell}_i + o(1)) \gamma_n \, , \hspace{2cm} 1 \leq i \leq r \, ,
 \end{align*}
where $(\rharpoonu{\ell}_i)_{i=1}^r$ are fixed, positive, and distinct parameters.  Correspondingly, we
study the normalized empirical eigenvalues
\begin{equation} \label{def-ulam}
\rharpoonu{\lambda}_{i} = \rharp \lambda_{i,n}  = \frac{ \lambda_{i}}{\gamma_n} \, .
\end{equation}
The next theorem provides the $\gamma_n \rightarrow \infty$
analogues of eigenvalue inflation (\ref{1}) and eigenvalue rotation (\ref{4}).

\begin{lemma} \label{thrm3}
(Benaych-Georges and Rao Nadakuditi \cite{BGN11}, Shen et al.\ \cite{Shen})
Under the disproportional limit $\gamma_n \rightarrow \infty$ and variable spikes {\normalfont \bf III},
the leading empirical eigenvalues of $S$ satisfy
\begin{equation} \phantom{\,.}
\rharpoonu{\lambda}_{i}  \xrightarrow{a.s.} \rharp 1 + \rharpoonu{\ell}_i \,.
\end{equation}
The angles between empirical eigenvectors and the corresponding population eigenvectors have limits
\begin{equation}
\begin{aligned} \phantom{\,.} \hspace{1cm}& | \langle u_i, v_j \rangle | \xrightarrow{a.s.}  \delta_{ij} \cdot \rharpoonu{c}(\rharpoonu{\ell}_i) \, ,\end{aligned} \qquad  \hspace{1cm} 1 \leq i, j \leq r  \,,
\end{equation}
where the cosine function is given by
\begin{align}
  \rharpoonu{c}^2(\rharpoonu{\ell})  =  \frac{\rharpoonu{\ell}}{1+\rharpoonu{\ell}} \, . \label{123}
\end{align}
\end{lemma}

No phase transition appears in this framing of the $\gamma_n \rightarrow \infty$
setting; for example, $\partial \rharpoonu{\lambda} / \partial \rharpoonu{\ell} = 1$ and $\partial \rharpoonu{c}/ \partial \rharpoonu{\ell} > 0$ for all $\rharpoonu{\ell} > 0$, while as $\gamma_n \rightarrow 0$, we had $\partial \tlam/\partial \overset{\leftharpoonup}{\ell} = 0$  and $\partial \overset{\leftharpoonup}{c} / \partial \overset{\leftharpoonup}{\ell} = 0$ for $0 < \overset{\leftharpoonup}{\ell} < 1$.

Recall that for $i \leq \min(n,p)$, $\lambda_i(X'X) = \lambda_i(XX')$; one might therefore expect that the phase transition as $\gamma_n \rightarrow 0$ would manifest here as well as a clear phase transition. Such a transition for the {\it eigenvalue} does occur under alternative scalings and coordinates to $\rharpoonu{\ell}$, $\rharpoonu{\lambda}$. Indeed, remaining in  the $\gamma_n \rightarrow \infty$ limit, consider $\tilde \ell_i = \tilde \ell_{i,n} = \gamma_n(1+\lharp \ell_i(1+o(1))\gamma_n^{-1/2})$. Leveraging $\lambda_i(X'X) = \lambda_i(XX')$ and earlier $\gamma_n \rightarrow 0$ results, a phase transition occurs at $\overset{\leftharpoonup}{\ell}_i = 1$. This transition, however, tells us nothing of the eigenvectors: the properties of eigenvectors of $X'X$ and $XX'$ are quite different, and on this scale, leading empirical eigenvectors are asymptotically decorrelated from their population counterparts. By adopting spikes {\bf III}, we work on a far coarser scale, one where eigenvectors correlate but with no visible phase transition.

\subsection{Asymptotic Loss and Unique Admissibility in the $\gamma_n \rightarrow \infty$ Limit}

Under variable spikes {\bf III}, the population covariance
now explodes. As losses similarly explode, we consider rescaled losses:
\[
\rharpoonu{L}_{\star, k}(\Sigma, \widehat \Sigma) =  \frac{L_{\star,k}(\Sigma,\widehat \Sigma)}{\gamma_n} \,   .
\]
Let $\rharpoonu{\phi}_n(\lambda) = \lambda / \gamma_n$ denote the mapping to this new coordinate system. Thus, spikes {\bf III} and (\ref{def-ulam}) may be written as $\rharpoonu{\lambda}_{i} = \rharpoonu{\phi}_n(\lambda_{i})$ and $\rharp \phi_n(\ell_i) \rightarrow \rharp \ell_i$.

\begin{definition}
Let $\eta = \eta_n$ denote a sequence of shrinkers, possibly varying with $n$. Suppose that under the disproportional limit $\gamma_n \rightarrow \infty$  and varying-spikes {\bf III}, the sequences of normalized shrinker outputs induced by rescaling converge:
\[   \phantom{\,.}   \rharpoonu{\phi}_n(\eta(\lambda_i)) \xrightarrow{a.s.} \rharp \eta_i     \, , \hspace{2cm} 1 \leq i \leq r \, .  \]
We call the limits $(\rharp \eta_i)_{i=1}^r$ the {\it asymptotic shrinkage descriptors}.
\end{definition}

\begin{lemma}
\label{lem-dpg-grow-asy-loss}
 Let $\eta$ denote a sequence of shrinkers with asymptotic shrinkage descriptors $(\rharp \eta_i)_{i=1}^r$ under the disproportional limit $\gamma_n \rightarrow \infty$ limit and varying-spikes {\normalfont \bf III}. Each loss $\rharp L_{\star,1}$ then converges almost surely to a deterministic limit:
\[
  \rharpoonu{L}_{\star,1}(\Sigma, \widehat{\Sigma}_{\eta})   \xrightarrow{a.s.}
  \rharpoonu{\cal L}_{\star}((\rharpoonu{\ell}_i)_{i=1}^r, (\rharpoonu{\eta}_i)_{i=1}^r) , \qquad \star \in \{ F,O,N\} \, .
\]
The asymptotic loss is  sum- or max- decomposable into $r$ terms involving matrix norms applied to the $2 \times 2$ matrices $\widetilde{A}$ and $\widetilde{B}$ introduced in Lemma \ref{lem-dpg-asy-loss}.
With $\rharpoonu{\ell}_i$ denoting a spiked eigenvalue and $\rharp c(\rharp \ell)$ the limiting cosine in  (\ref{123}),
the decompositions are
 \begin{align*} \phantom{\,,} \rharpoonu{\cal L}_{F}((\rharpoonu{\ell}_i)_{i=1}^r, (\rharpoonu{\eta})_{i=1}^r) & = \bigg( \sum_{i=1}^r \big[ L_{F, 1} \big(\widetilde{A}(\rharpoonu{\ell}_i),\widetilde{B}(\rharpoonu{\eta}_i,\rharpoonu{c}(\rharp \ell_i)) \big) \big]^2 \bigg)^{1/2} \,, \\
 \rharpoonu{\cal L}_{O}((\rharpoonu{\ell}_i)_{i=1}^r,  (\rharpoonu{\eta})_{i=1}^r) & = \max_{1 \leq i \leq r} L_{O,1} \big(\widetilde{A}(\rharpoonu{\ell}_i),\widetilde{B}(\rharpoonu{\eta}_i,\rharp c(\rharp \ell_i))\big) \, , \\
 \rharpoonu{\cal L}_{N}((\rharpoonu{\ell}_i)_{i=1}^r,  (\rharpoonu{\eta})_{i=1}^r) & = \sum_{i=1}^r L_{N,1} \big(\widetilde{A}(\rharpoonu{\ell}_i),\widetilde{B}(\rharpoonu{\eta}_i, \rharp c(\rharp \ell_i)) \big) \, .
\end{align*}
 \end{lemma}

 Proof of this lemma is similar to that of Lemma  \ref{lem-dpg-asy-loss} and omitted. Note that only pivot $\Delta_1$ is considered as $S$ and $\widehat \Sigma_\eta$ have $p - n$ eigenvalues equal to zero. As a simple example, the asymptotic shrinkage descriptors of the  rank-aware sample covariance estimator $S^r= \sum_{i=1}^r (\lambda_i - 1) v_i v_i' + I$ are $\rharp \eta_i = \rharp \lambda(\rharp \ell_i)$.
Squared asymptotic loss evaluates to (suppressing the subscript of $\rharp \ell_1$)
\begin{equation} \label{eq-dpg-grow-asy-loss-fro}
  [\rharpoonu{\cal L}_{F}(\rharpoonu{\ell}, \rharpoonu{\lambda})]^2 =  (\rharpoonu{\ell} - \rharpoonu{\lambda} \rharpoonu{c}^2(\rharp \ell))^2 + \rharpoonu{\lambda}^2 ( 1 - \rharpoonu{c}^4(\rharp \ell))  \, .
\end{equation}
By Theorem \ref{thrm:spiked_covar2},  $\rharpoonu{\lambda} \rharpoonu{c}^2(\rharp \ell) = \rharpoonu{\ell}$,
while $\rharpoonu{\lambda}^2 (1 - \rharpoonu{c}^4(\rharp \ell)) = (1+2 \rharpoonu{\ell})$, so $[\rharpoonu{\cal L}_{F}(\rharpoonu{\ell}, \rharpoonu{\lambda})]^2 = (1+2\rharpoonu{\ell})$. Asymptotic losses of $S^1$ under each norm are collected below in Table 3, to later facilitate comparison with optimal shrinkage.
\setlength\extrarowheight{4pt}
 \begin{table}[h!]
 \centering
 \begin{tabular}{| c | c |}
\hline
 Norm &  $\rharp {\cal L}_\star(\rharp \ell, \rharp \lambda)$ \\
\hline
Frobenius &  $\sqrt{1 + 2 \rharpoonu{\ell}}$ \\
Operator &  $\left (1 + \sqrt{1+4 \rharpoonu{\ell}} \right )/2$ \\
Nuclear &  $\sqrt{1+4\rharpoonu{\ell}}$ \\
\hline
 \end{tabular}
 \caption{Asymptotic Loss $\rharpoonu{\cal L}_\star$ of the standard rank-aware estimator $S^1$.}
 \label{tbl-asy-loss-gro-rank-aware}
 \end{table}
\setlength\extrarowheight{0pt}

The intermediate form (\ref{eq-dpg-grow-asy-loss-fro}) is symbolically isomorphic
to the intermediate form   (\ref{eq-dpg-shr-asy-loss-fro}) seen earlier
in the $\gamma_n \rightarrow 0$ case (under replacement of $\overset{\leftharpoonup}{\;}$'s by $\overset{\rightharpoonup}{\;}$'s),
suggesting that the path to optimality will again lead
to eigenvalue shrinkage.

\begin{definition}
\label{def-gro-formal-asy}
In the $\gamma_n \rightarrow \infty$ limit, the {\it formally optimal asymptotic loss} in the rank-1 setting is
\[
\rharpoonu{\cal L}_{\star}^1(\rharpoonu{\ell})  \equiv \min_{\vartheta} \rharpoonu{\cal L}_{\star}(\rharpoonu{\ell},\vartheta), \qquad \star \in \{F,O,N\}.
\]
A {\it formally optimal shrinker} is a function $\rharpoonu{\eta}(\cdot | \star): \mathbb{R} \mapsto \mathbb{R}$
achieving $\rharpoonu{\cal L}_{\star}^1(\rharpoonu{\ell})$:
\[
     \rharpoonu{\eta}(\rharpoonu{\ell} | \star) = \underset{\vartheta}{\mbox{argmin}} \, \rharpoonu{\cal L}_{\star}(\rharpoonu{\ell},\vartheta), \qquad \rharpoonu{\ell} > 0, \qquad \star \in \{F,O,N\}.
\]
\end{definition}

In complete analogy with Lemma \ref{lem-shr-opt}, we have explicit forms of formally optimal shrinkers.
\begin{lemma}
\label{lem-gro-opt}
Formally optimal shrinkers (defined analogously to Definition \ref{def48}) and corresponding losses are given by
\begin{align} \label{eval-gro-oper-loss}
    & \rharpoonu{\eta}^*(\rharpoonu{\ell}|F)  = \frac{\rharpoonu{\ell}^2}{1+\rharpoonu{\ell}} \,,&&   [\rharpoonu{\cal L}_{F}^1(\rharpoonu{\ell})]^2 =  \frac{\rharpoonu{\ell}^2 (2\rharpoonu{\ell} +1)}{(\rharpoonu{\ell}+1)^2}\,, \nonumber  \\
    & \rharpoonu{\eta}^*(\rharpoonu{\ell}|O) = \rharpoonu{\ell}  \,,&&\rharpoonu{\cal L}_{O}^1(\rharpoonu{\ell}) =  \frac{ \rharpoonu{\ell}}{(1+\rharpoonu{\ell})^{1/2}} \,, \\
    & \rharpoonu{\eta}^*(\rharpoonu{\ell}|N) = \rharpoonu{\ell} \bigg( \frac{\rharpoonu{\ell}-1}{\rharpoonu{\ell}+1} \bigg)_+ && \rharpoonu{\cal L}_{N}^1(\rharpoonu{\ell}) = \rharpoonu{\ell} \cdot  \bigg[ 1 _{\{\rharpoonu{\ell} < 1\}} +  1_{\{\rharpoonu{\ell} > 1\}} \cdot \frac{2 \cdot \sqrt{\rharpoonu{\ell}}}{\rharpoonu{\ell}+1} \bigg]\,. \nonumber
    \end{align}
\end{lemma}
\begin{proof}
Asymptotic losses are functions of the limiting formulas for eigenvalue inflation and eigenvector rotation. Thus, by the proof Lemma \ref{lem-shr-opt}, in particular (\ref{9237}),
\begin{align}
 &    \rharpoonu{\eta}^*(\rharpoonu{\ell} | F) = \rharpoonu{\ell}   \rharpoonu{c}^2(\rharpoonu{\ell}) \, , &   \rharpoonu{\eta}^*(\rharpoonu{\ell} | O) = \rharpoonu{\ell} \, , &&   \rharpoonu{\eta}^*(\rharpoonu{\ell} | N) = \rharpoonu{\ell} (1-2  \rharpoonu{s}^2(\rharpoonu{\ell}))_+ \, .
\end{align}
Substitution of (\ref{123}) yields the left-hand column of (\ref{eval-gro-oper-loss}). In parallel fashion, asymptotic losses are isomorphic:
\[   \lharpoonu{\cal L}_F^*(\overset{\leftharpoonup}{\ell}) = \overset{\leftharpoonup}{\ell}^2 \overset{\leftharpoonup}{s}^2 (2 -\overset{\leftharpoonup}{s}^2) \, , \qquad \rharpoonu{\cal L}_F^*(\rharpoonu{\ell}) = \rharpoonu{\ell}^2 \rharpoonu{s}^2 (2 -\rharpoonu{s}^2) \, . \]
\end{proof}

\begin{theorem} \label{10}
Define the following shrinkers through the formally optimal shrinkers $\rharpoonu{\eta}(\rharp \ell | \star)$ of Lemma \ref{lem-gro-opt}:
\begin{align*}
 \phantom{\,.} \eta^*(\lambda | \star) = \eta_n^*(\lambda | \star) &= \rharp \phi_n^{-1} \big( \rharpoonu{\eta}( \rharp \phi_n(\lambda) - 1| \star ) \big) \\
&= \gamma_n \rharp \eta ( \lambda / \gamma_n - 1 | \star)\,. \end{align*}
Under $\gamma_n \rightarrow \infty$ and variable spikes {\normalfont \bf III}, $\eta^*(\cdot | \star)$ achieves the optimal asymptotic normalized loss. Up to asymptotically negligible perturbations, $\eta^*(\cdot| \star)$ is the unique asymptotically admissible shrinker for  $\rharp L_{\star,1}$.
\end{theorem}

All formally optimal shrinkers are continuous. The proof of Theorem \ref{10} is analogous to that of Theorem \ref{thrm:spiked_covar2} and is omitted.

\begin{corollary}
Under $\gamma_n\rightarrow \infty$ and variable spikes {\normalfont \bf III}, both the sample covariance $S$ and the rank-aware sample covariance $S^r$ are asymptotically inadmissible for $L_{\star,1}$.
 \end{corollary}

\subsection{Performance in the $\gamma_n \rightarrow \infty$ Limit}

Figure \ref{fig-gro-optshrink} depicts optimal shrinkers (left) and corresponding asymptotic losses (right, in the rank-one case $r=1$), similarly to Figure \ref{fig-shr-optshrink}.
In the left-hand panel, the red curve again marks no shrinkage, $\rharpoonu{\eta}(\rharpoonu{\lambda}) = \rharpoonu{\lambda}$. Each optimal shrinker $\rharpoonu{\eta}^*(\cdot| \star)$ lies below the diagonal---especially for small $\rharpoonu{\lambda}$. Normalized (non-zero) eigenvalues converge to $\rharpoonu{\lambda} = 1$;  optimal shrinkers all vanish below $\rharpoonu{\lambda} \leq 1$.

The right-hand panel compares performances under various loss functions of the standard estimator $S^r$ (dotted lines) and the optimal estimators (solid lines). Asymptotic losses of the standard estimator are strictly larger than that of optimal estimators for all $\rharp \ell$. Under Frobenius and nuclear norms, optimal shrinkage also outperforms the fixed shrinker $\eta(\lambda) = \lambda - 1$. As $\rharpoonu{\ell} \rightarrow 0^+$, optimal
losses $\rharpoonu{\cal L}_\star^*(\rharpoonu{\ell})$ tend to zero,
while standard losses tend to 1.
The maximal relative regret for  $S^r$ is unbounded.

For example, with operator norm loss, $\rharpoonu{\cal L}_O(1,\rharpoonu{\lambda}) = (1+\sqrt{5})/2$, while $\rharpoonu{\cal L}_O(1,\rharpoonu{\eta}^*)=1/\sqrt{2}$.
The absolute regret is  $\rharp {\mathcal{R}}_O (1,\rharpoonu{\lambda})  =  .91$,
and 57\% improvement in loss is possible at $\rharpoonu{\ell}=1$.
The maximal possible relative improvement is 100\%:
at $\rharpoonu{\ell}=0$, all the loss incurred by the standard estimator is avoidable.
Under Frobenius norm, $\rharpoonu{\cal L}_F(1,\rharpoonu{\lambda})=\sqrt{3}$,
$\rharpoonu{\cal L}_F(1, \rharp \eta^*) = \sqrt{3}/2$, and $\rharp {\mathcal{R}}_F(1,\rharpoonu{\lambda}) = \sqrt{3}/2$. There is 50\% possible improvement over the standard estimator at $\rharpoonu{\ell}=1$, and fully $100$\% of the standard loss is avoidable using shrinkage at $\rharpoonu{\ell}=0$.

\setlength\extrarowheight{4pt}
\begin{table}[h]
\centering
\begin{tabular}{| c | c | c | c | c |}
\hline
Norm & $\rharp {\mathcal{R}}_\star(0^+,\rharpoonu{\lambda})$ & $\rharp {\mathcal{R}}_\star(1,\rharpoonu{\lambda})$ & $\rharp {\cal I}_\star(0^+,\rharpoonu{\lambda})$ & $\rharp {\cal I}_\star(1,\rharpoonu{\lambda})$\\
\hline
Frobenius & 1 & $\sqrt{3}/2$ &  100\% & 50\%  \\
Operator &  1  &  2.52 & 100\% & 57\%  \\
Nuclear &  1 &   $\sqrt{5}-1$  & 100\% & 56\%  \\
\hline
\end{tabular}
\caption{{\bf Regret and Improvement, $\gamma_n \rightarrow \infty$.}
Absolute Regret  $\rharp {\mathcal{R}}$ and possible relative improvement  $\rharp {\cal I}$ of
the standard rank-aware estimator $S^r$ (equivalently, $\rharpoonu{\eta} = \rharpoonu{\lambda}$) near zero and exactly at  $\rharpoonu{\ell}  = 1$.}
\end{table}

\begin{figure}[h!]
\centering
\includegraphics[height=2.6in]{spiked_covar4.png} \hspace{-3cm}
\includegraphics[height=2.6in]{spiked_covar5.png}
\caption{{\bf Optimal shrinkers and losses, $\gamma_n \rightarrow \infty$.}
Left: optimal shrinkage functions. Right: losses of optimal shrinkers (solid) and of the standard estimator $S^r$, under Frobenius (blue), operator (orange), and nuclear (green) norms.}
\label{fig-gro-optshrink}
 \end{figure}
\setlength\extrarowheight{0pt}

\begin{figure}[h!]
\centering
\includegraphics[height=2.5in]{spiked_covar8}
\caption{{\bf Monte-Carlo simulations, large $\gamma_n$.}
Averages over 50 realizations of losses under three norms
for the standard and asymptotically optimal estimators. Here, $p=10{,}000$ and $n = 100$, so $\gamma_n= 100$.}
\label{fig-gro-empirical}
 \end{figure}

\section{Optimal Hard Thresholding}
\label{sec-opt-hard-thresh}

A natural alternative to optimal shrinkage is simple {\it hard thresholding}, applying the zero-based shrinker $H_\tau(\theta) = \theta \cdot 1_{\{\theta \geq \tau\}} + 1_{\{ \theta < \tau \}}$.
In original coordinates,
$\widehat{\Sigma}_{H_\tau} = \sum_{i=1}^r H_{\tau_n}(\lambda_i) v_i v_i' + I$;
in rescaled coordinates, this corresponds to hard thresholding the normalized eigenvalues at the below levels:
\begin{itemize}
 \item $\lharpoonu{\tau}_n = \lharpoonu{\phi}_n(\tau_n)$,
 i.e. $H_{\lharpoonu{\tau}_n}(\tlam_j)$ ($\gamma_n  \rightarrow 0$),
 \item $\rharpoonu{\tau}_n = \rharpoonu{\phi}_n(\tau_n)$,
 i.e. $H_{\rharpoonu{\tau}_n}(\rharpoonu{\lambda}_j)$  ($\gamma_n  \rightarrow \infty$).
 \end{itemize}
 Under the appropriate varying-spikes model {\bf II} or {\bf III},
 it makes sense to choose threshold sequences
 $\tau_n$ that, after normalization, are essentially constant:
 \begin{itemize}
 \item  $\lharpoonu{\tau}_n = \lharpoonu{\tau} \cdot ( 1+ o(1))$ ($\gamma_n  \rightarrow 0$),
 \item   $\rharpoonu{\tau}_n = \rharpoonu{\tau} \cdot (1 + o(1))$ ($\gamma_n  \rightarrow \infty$).
\end{itemize}
It may seem natural to place the threshold exactly {\it at} the bulk edge. We propose, however, a performance-dominating alternative, with thresholds notably beyond the bulk edge. See Table \ref{tbl-opt-thresh}.

 \setlength\extrarowheight{2pt}
 \begin{table}[h!]
 \centering
 \begin{tabular}{| c | c | c |}
\hline
 Norm & $\lharpoonu{\tau}(\star)$ & $\rharpoonu{\tau}(\star)$ \\
\hline
Frobenius & $4/\sqrt{3}$ & $1+\sqrt{2}$ \\
Operator & $\sqrt{2 (1 + \sqrt{2})}$& $3$ \\
Nuclear & $6 / \sqrt{5}$ & $3+\sqrt{5}$ \\
\hline
\hline
Bulk Edge & 2 & 1 \\
\hline
 \end{tabular}
 \caption{{\bf Optimal thresholding parameters.} Thresholds  in rows 2 through 4 are considerably beyond the bulk edge in row 5. To use these (normalized) thresholds with unnormalized eigenvalues,
 back-translate:  use  $\tau_n = \lharpoonu{\phi}_n^{-1}(\lharpoonu{\tau})$ as $\gamma_n \rightarrow 0$ and $\tau_n = \rharpoonu{\phi}_n^{-1}(\rharpoonu{\tau})$ as  $\gamma_n \rightarrow \infty$.}
 \label{tbl-opt-thresh}
 \end{table}
 \setlength\extrarowheight{0pt}

 \begin{definition}
We say that $\lharpoonu{\tau}$ is the {\it unique admissible  normalized threshold} for asymptotic loss   $\lharpoonu{\cal L}_\star( \lharp \ell, \cdot)$ as $\gamma_n \rightarrow 0$ if,  for any other deterministic normalized threshold $\lharpoonu{\nu}$, we have
 \[
  \lharpoonu{\cal L}_\star( \overset{\leftharpoonup}{\ell}, H_{\lharpoonu{\tau}}) \leq \lharpoonu{\cal L}_\star( \overset{\leftharpoonup}{\ell}, H_{\lharpoonu{\nu}}) \, , \hspace{2cm} \forall \, \lharp \ell \geq 0 \, ,
 \]
 with strict inequality at some $\overset{\leftharpoonup}{\ell}' \geq 0$. We similarly define the unique admissible normalized threshold $\rharp \tau$ for $\rharp {\cal L}_\star(\rharp \ell,\cdot)$ as $\gamma_n \rightarrow \infty$.
 \end{definition}

 \begin{theorem}
 \label{thm-opt-thresh}
 For $\star \in \{ F,O,N\}$, there are unique admissible
 thresholds $\lharpoonu{\tau}(\star)$ and $\rharpoonu{\tau}(\star)$  for  asymptotic losses $\lharpoonu{\cal L}_ \star( \overset{\leftharpoonup}{\ell}, \overset{\leftharpoonup}{\eta})$ and $\rharpoonu{\cal L}_\star( \rharpoonu{\ell}, \rharpoonu{\eta})$, respectively, given in Table \ref{tbl-opt-thresh}.
 \end{theorem}

\begin{proof}
Consider $\gamma_n \rightarrow 0$. $\overset{\leftharpoonup}{\ell} \mapsto \lharpoonu{\cal L}(\overset{\leftharpoonup}{\ell},0)$ and $\overset{\leftharpoonup}{\ell} \mapsto \lharpoonu{\cal L}(\overset{\leftharpoonup}{\ell},\tlam)$ denote the asymptotic losses of the null $\overset{\leftharpoonup}{\eta}(\tlam)=0$ and identity $\overset{\leftharpoonup}{\eta}(\tlam) = \tlam$ rules, respectively. In each case of Table \ref{tbl-opt-thresh}, there is an unique crossing point $\lharpoonu{\theta}(\star)$ exceeding the bulk edge such that
\[
   \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},0) <  \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\tlam) \, ,  \quad \overset{\leftharpoonup}{\ell} < \lharpoonu{\theta}(\star) \, ; \qquad
    \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},0) <   \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\tlam)\, ,  \quad \overset{\leftharpoonup}{\ell} > \lharpoonu{\theta}(\star) \, .
\]
Equality occurs only for $\overset{\leftharpoonup}{\ell} = \lharpoonu{\theta}(\star)$. Calculations of $\lharp \theta(\star)$ are omitted. Define $\lharpoonu{\tau}(\star) = \tlam(\lharpoonu{\theta}(\star))$. Note that
\begin{align*} & H_{\lharpoonu{\tau}(\star)}(\tlam) \xrightarrow[]{a.s.} 0 \, ,  & \overset{\leftharpoonup}{\ell} < \lharpoonu{\theta} \, , \\
& H_{\lharpoonu{\tau}(\star)}(\tlam) \xrightarrow{a.s.} \tlam(\overset{\leftharpoonup}{\ell}) \, , & \overset{\leftharpoonup}{\ell} > \lharpoonu{\theta} \, .
\end{align*}
Consequently,
\[
              \lharp L_{\star,k}(\overset{\leftharpoonup}{\ell}, H_{\tau(\star)}) \xrightarrow{a.s.} \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},H_{\tau(\star)})  = \min \big(   \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},0) ,  \, \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\tlam) \big) \, .
\]
Let $\lharpoonu{\nu}$ denote another choice of threshold. Now, for every $\overset{\leftharpoonup}{\ell}$,
\[
\lharpoonu{\cal L}_\star( \ell, { H}_{\lharpoonu{\nu}}) \in \big\{  \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},0) ,  \,  \lharpoonu{\cal L}_\star(\overset{\leftharpoonup}{\ell},\tlam) \big\}.
\]
The loss $\lharpoonu{\cal L}_\star( \overset{\leftharpoonup}{\ell},H_{\lharpoonu{\tau}(\star)})$ is the minimum of these two. Hence, for every $\overset{\leftharpoonup}{\ell}$,
\begin{equation} \label{eq-not-worse}
  \lharpoonu{\cal L}_\star( \overset{\leftharpoonup}{\ell},H_{\lharpoonu{\tau}(\star)}) \leq \lharpoonu{\cal L}_\star( \overset{\leftharpoonup}{\ell},H_{\lharpoonu{\nu}}) \, .
\end{equation}
Since $\lharpoonu{\nu} \neq \lharpoonu{\tau}(\star)$, there is
an intermediate value $\lharpoonu{\theta}'$ between $\lharpoonu{\theta}$ and $\overset{\leftharpoonup}{\ell}(\lharpoonu{\nu})$
such that $\tlam(\lharpoonu{\theta}')$  is  intermediate between  $\lharpoonu{\tau}(\star)$ and $\lharpoonu{\nu}$.
At $\lharpoonu{\theta}'$, of the two procedures behaves as the null rule while the other behaves as the identity. The two asymptotic loss functions cross only at a single point $\lharpoonu{\theta}(\star)$. Hence, at $\lharpoonu{\theta}'$ the asymptotic loss functions are unequal. By (\ref{eq-not-worse}),
\begin{equation} \label{eq-strict-better} \phantom{\,.}
\lharpoonu{\cal L}_\star(\lharpoonu{\theta}',H_{\lharpoonu{\tau}(\star)})  < \lharpoonu{\cal L}_\star(\lharpoonu{\theta}',H_{\lharpoonu{\nu}}) \, .
\end{equation}
Together, (\ref{eq-not-worse}) and (\ref{eq-strict-better})  establish unique asymptotic admissibility. The argument as $\gamma_n \rightarrow \infty$ is similar.
\end{proof}

 Figure \ref{fig-loss-crossing} depicts two of the six cases:
Frobenius norm as $\gamma_n \rightarrow 0$, and  nuclear norm as $\gamma_n \rightarrow \infty$.

 \begin{figure}[h!]
\centering
\includegraphics[height=2.2in]{OptThreshFrobenius.png}
\includegraphics[height=2.2in]{OptThreshNuc.png} \\
\caption{{\bf Determining the optimal threshold}.
Left: Frobenius norm,  $\gamma_n \rightarrow 0$.
The two loss functions $\overset{\leftharpoonup}{\ell} \mapsto \lharpoonu{\cal L}_F(\overset{\leftharpoonup}{\ell},0)$, $\overset{\leftharpoonup}{\ell} \mapsto \lharpoonu{\cal L}_F(\overset{\leftharpoonup}{\ell},\tlam)$
cross in a single point $\overset{\leftharpoonup}{\ell}=\lharpoonu{\theta}(F)$. The optimal threshold is  $\lharpoonu{\tau} = \tlam(\lharpoonu{\theta}(F))$.
Right: nuclear norm, $\gamma_n \rightarrow \infty$.}
\label{fig-loss-crossing}
 \end{figure}

 \section{Universal Closed Forms}
\label{sec-uni-clo}

The spiked covariance model poses a
``scaling dilemma''  for practitioners:

\begin{quotation}
\sl I only have my one dataset, with its own specific $p$ and $n$.
I don't know what asymptotic scaling $(n, p)$  my dataset ``obeys." Yet, I have two theories
 seemingly competing for my favor: proportional and
 disproportional growth. Each theory has its own optimal formulas. Which should I apply?
\noindent Fortunately, this dilemma can be avoided.
\end{quotation}

\begin{definition}
Let $\eta^+(\lambda | L,  \gamma)$ denote a closed-form shrinker for the {\it proportional}  growth regime, mentioned following Lemma \ref{lem:dg18_7}.
Given a dataset of size $n \times p$, define the {\it universal shrinker}
\[ \phantom{\,.}
\eta^u(\lambda | L) =  \eta_n^{u}(\lambda | L )  = \eta^+(\lambda | L, p/n) \, .
\]
That is, we evaluate $\eta^+$ using the aspect ratio $\gamma_n = p/n$ of the given dataset.
This requires {\it no hypothesis} on scaling of $p$ with $n$.
We also denote by $\widehat{\Sigma}^u$ the shrinkage estimator $\widehat{\Sigma}_{\eta^u}$.
\end{definition}

\begin{observation}  Adopt loss $L = L_{\star,k}$ for $\star \in \{ F,O,N\}$. The asymptotic shrinkage descriptors of the universal shrinker $\eta^u(\lambda| L)$ are optimal in the proportional and disproportional limits.
\end{observation}

\begin{enumerate}
\item  Assume the proportional limit $\gamma_n \rightarrow \gamma$. The asymptotic shrinkage descriptors, or shrinkage limits, of the optimal proportional-regime rule $\eta^+(\lambda | L, \gamma)$ are
\[ \phantom{\,.}
\eta_i^+ =\lim_{n \rightarrow \infty} \eta^+(\lambda_{i} | L, \gamma)
\, .\]
The corresponding  shrinkage limits
\[
\eta_i^u =\lim_{n \rightarrow \infty} \eta^u(\lambda_{i} | L, \gamma_n)  = \lim_{n \rightarrow \infty} \eta^+(\lambda_{i} | L, \gamma_n)
 \]
almost surely exist and are identical:
\[
    \eta_i^u \stackrel{a.s.}{=}  \eta_i^+ \, ,  \qquad i=1,\dots,r \, .
\]
The  asymptotic losses of the two shrinkers as calculated by Lemma  \ref{lem:dg18_7} are almost surely identical.

\item Assume the disproportional limit $\gamma_n \rightarrow 0$ and varying spikes {\normalfont \bf II}. The shrinkage limits of $\eta^u$ and $\eta^*$ are
\[
\overset{\leftharpoonup}{\eta}_i^u = \lim_{n \rightarrow \infty} \lharpoonu{\phi}_n(\eta^u(\lambda_{i} | L)) \, ,
\hspace{2cm} \overset{\leftharpoonup}{\eta}_i^* =   \lim_{n \rightarrow \infty} \overset{\leftharpoonup}{\eta}^*(\lharpoonu{\phi}_n(\lambda_{i}) | L) \, . \]
These limits almost surely exist and are identical:
\[
\phantom{\,.} \overset{\leftharpoonup}{\eta}_i^u \stackrel{a.s.}{=}   \overset{\leftharpoonup}{\eta}_i^* \, ,   \qquad i=1,\dots,r  \, . \]
The asymptotic losses of the two shrinkers as calculated by Lemma  \ref{lem-dpg-asy-loss} are almost surely identical.

\item Assume the  disproportional limit $\gamma_n \rightarrow \infty$ and varying spikes {\normalfont \bf III}. The shrinkage limits of $\eta^u$ and $\eta^*$ are
\[
\rharpoonu{\eta}_i^u = \lim_{n \rightarrow \infty} \rharpoonu{\phi}_n(\eta^u(\lambda_{i} | L)) \, , \hspace{2cm}
\rharpoonu{\eta}_i^* = \lim_{n \rightarrow \infty} \rharpoonu{\eta}^*(\rharpoonu{\phi}_n(\lambda_{i}) | L ) \, .
\]
These limits almost surely exist and are identical:
\[
    \rharpoonu{\eta}_i^u  \stackrel{a.s.}{=}  \rharpoonu{\eta}_i^* \, ,  \qquad i=1,\dots,r \, .
\]
The asymptotic losses of the two shrinkers as
calculated by Lemma  \ref{lem-dpg-grow-asy-loss} are almost surely  identical.
\end{enumerate}

For example, recall the proportional-regime shrinker for $L_{F,1}$:
\begin{align} \phantom{\,.}  \eta^+(\lambda | L_{F,1},\gamma) = 1+ (\ell(\lambda, \gamma)-1) c^2(\ell(\lambda, \gamma), \gamma)  \, . \label{9267}
\end{align}
Note that
\[ \phantom{\,.} \lharp \phi_n(\ell(\lambda, \gamma_n)) =  \frac{\lharp \phi_n(\lambda) - \sqrt{\gamma_n} + \sqrt{(\lharp \phi_n(\lambda) - \sqrt{\gamma_n})^2 - 4}}{2} = \lharp \ell (\lharp \phi_n(\lambda) - \sqrt{\gamma_n}) \, ,
\]
so $\lharp \phi_n(\ell(\lambda_i, \gamma_n))  \xrightarrow{a.s.} \overset{\leftharpoonup}{\ell} (\tlam_i)$. Thus,
\begin{align*} \phantom{\,.} \lharp \phi_n( \eta^u(\lambda_i | L_{F,1})) & =\frac{\ell(\lambda_i, \gamma_n) - 1}{\sqrt{\gamma_n}} \cdot c^2(\ell(\lambda_i, \gamma_n ), \gamma_n) \\
&= \lharp \phi_n(\ell(\lambda_i, \gamma_n)) \cdot \frac{1 - 1 / \phi_n^2(\ell(\lambda_i, \gamma_n))}{1 + \sqrt{\gamma_n} / \phi_n(\ell(\lambda_i, \gamma_n))} \cdot 1_{\{\tlam_i > 1\}}\\
&\xrightarrow{a.s.} (\overset{\leftharpoonup}{\ell}_i - 1/\overset{\leftharpoonup}{\ell}_i)_+ \, ,
\end{align*}
agreeing with Lemma \ref{lem-shr-opt}.

 \begin{corollary}\label{cor2}
The estimator sequence  $\widehat{\Sigma}_{\eta^u}$ is
uniquely asymptotically admissible under either
the proportional or disproportional growth regimes.
 \end{corollary}

This principle applies more broadly; consider thresholding. Constructed in the previous section as $\gamma_n \rightarrow 0$ and $\gamma_n \rightarrow \infty$, optimal thresholds also exist in the proportional limit $\gamma_n \rightarrow \gamma \in (0,\infty)$. These three choices of threshold, depending on the limit regime, again present a scaling conundrum to practitioners.

One can easily check, however, that under each loss there exists a simple close-form threshold which performs optimally in all three limits. Under proportional growth and $L_{F,1}$, the losses of the null and identity rules are
\[
 {\cal L}_{F,1}( \ell,0) = (\ell-1)^2 \, , \qquad {\cal L}_{F,1}(\ell,\lambda) = (\ell - \lambda(\ell,\gamma))^2 + (\lambda(\ell,\gamma)-1)^2(1 -c^4(\ell,\gamma)) \, .
\]
$\ell \mapsto {\cal L}_{F,1}(\ell,0)$ is increasing and, for $\ell > \ell_+(\gamma)$, $\ell \mapsto {\cal L}_{F,1}(\ell,\lambda)$ is decreasing.
The crossing point $\theta_\gamma(F,1)$ of the two losses ${\cal L}_{F,1}(\theta_\gamma,0) = {\cal L}_{F,1}(\theta_\gamma, \lambda)$ can be shown to be the largest real root of
\[
\gamma^3  \theta^2 + 3 \gamma^2 \theta^2 (\theta-1) + \gamma (\theta+1) (\theta-1)^3  =  (\theta-1)^5.
\]
The corresponding threshold is $\lambda_\gamma(F,1) = \lambda(\theta_\gamma(F,1),\gamma)$.
One may verify that
\[
 \lambda_\gamma(F,1) \sim 1 + \sqrt{\gamma} \cdot \lharpoonu{\tau}(F) \, , \quad \gamma \rightarrow 0 \, ;  \qquad
 \lambda_\gamma(F,1) \sim 1 +  \gamma \cdot \rharpoonu{\tau}(F) \, , \quad \gamma  \rightarrow \infty \, .
\]
Define the {\it universal threshold rule} $\lambda^u(F,1) = \lambda^u_n(F,1)$ by
evaluating the proportional rule with the aspect ratio $\gamma_n = p/n$ of the given dataset: $\lambda^u(F,1) = \lambda_{\gamma_n}(F,1)$. This threshold can be applied as is---it requires no scaling hypothesis. Nevertheless, it is an optimal threshold in both the proportional limit and either disproportional limit.

\section{Estimation in the Spiked Wigner model}

We now develop a connection to the {\it spiked Wigner model}.
Let $W = W_n$ denote a {\it Wigner matrix}, a real symmetric matrix of
size $n \times n$ with independent entries on the upper triangle
distributed as $\mathcal{N}(0,1)$. The empirical distribution of eigenvalues of $W$ converges (weakly almost surely) to $\omega(x) = (2\pi)^{-1} \sqrt{4 - x^2}$, the standard semicircle density with support $\overline{\lambda}_{\pm} = \pm 2$.

Let  $\Theta = \Theta_n$ denote a symmetric $n \times n$
``signal'' matrix of fixed rank $r$;
 under the {spiked Wigner model} observed data  $Y = Y_n$ obeys
\begin{align}
\phantom{\,. }Y = \Theta + \frac{1}{\sqrt{n}} W     \,.
\end{align}
Let $\theta_1 \geq \cdots \geq \theta_{r_+} > 0 > \theta_{r_++1} \geq \dots  \geq \theta_{r}$ denote the non-zero eigenvalues of $\Theta$, so there are $r_+$ positive values and $r_- = r - r_+$ negative, and $u_1, \ldots, u_n$ the corresponding eigenvectors. The standard (rank-aware) reconstruction is
\[ \phantom{\,.}
\widehat \Theta^{r} = \sum_{i=1}^{r_+}  \lambda_i(Y) v_i v_i'  +  \sum_{i=n-r_-+1}^n  \lambda_i(Y) v_i v_i'\, ,
\]
where $\lambda_1(Y) \geq \cdots \geq \lambda_n(Y)$ are the eigenvalues of $Y$
and $v_1, \ldots, v_n$ the associated eigenvectors.

Maïda \cite{Maida2007},
Capitaine, Donati-Martin and Feral \cite{Capitaine2009}, and Benaych-Georges and Rao Nadakuditi \cite{BGN11},  among others, derive an eigenvalue mapping $\overline{\lambda}(\theta)$ describing the
empirical eigenvalues induced by signal eigenvalues $\theta_i$. Their results imply
that the top $r_+$ empirical eigenvalues of $Y$ obey   $\lambda_i(Y)  \xrightarrow{a.s.} \overline{\lambda}(\theta_i)$, $i=1,\dots,r_+$,
while the lowest $r_-$ obey $\lambda_{n-i}  \xrightarrow{a.s.} \overline{\lambda}(\theta_{r-i})$, $0 \leq i < r_-$. Here the eigenvalue mapping function is defined by
\begin{align} \phantom{\,.}  \overline{\lambda}(\theta) = \begin{dcases}
      \theta + \frac{1}{\theta}  & |\theta| > 1\\
      2 \, \text{sign}(\theta) &  0 < |\theta| \leq  1
   \end{dcases}  ,  \end{align}
with phase transitions at $\pm 1$ mapping to bulk edges $\overline{\lambda}_\pm = \pm 2$. There is a partial inverse to $\theta \mapsto \overline{\lambda}(\theta)$:
\begin{equation} \label{eq:theta-def}
\theta(\lambda) = \begin{dcases}
(\lambda + \text{sign}(\lambda) \sqrt{ \lambda^2 - 2 \sigma^2})/2 & |\lambda| > 2 \\
 0 &  | \lambda | \leq  2
\end{dcases} \, .
 \end{equation}

Empirical eigenvectors are not perfectly aligned with the corresponding signal eigenvectors:
\[ |\langle u_i, v_i \rangle |^2 \xrightarrow{a.s.} \overline c^2(\theta_i)  \, , \qquad i \in \{1, \ldots, r_+,  n-r_-+1 ,  \ldots , n\} \, ,
\]
where the cosine function is given by
\begin{align} \phantom{\,.}   \overline{c}^2(\theta) = \begin{dcases}
       1 - \frac{1}{\theta^{2}} & |\theta| > 1\\
      0 & \theta \leq 1
   \end{dcases} .  \label{wig_cos}
\end{align}
The phenomena of spreading, inflation, and rotation
imply that  $\widehat \Theta^r$ can be improved upon,
substantially, by well-chosen shrinkage estimators:
\begin{align}
     \widehat{\Theta}_\eta  = \sum_{i=1}^n \eta(\lambda_i(Y)) v_i v_i' \, ,
\end{align}
with $\eta: \mathbb{R} \rightarrow \mathbb{R}$ a shrinkage function. For numerous loss functions
 $L$, specific shrinkers $\eta^*( \cdot | L)$
outperform the standard rank-aware estimator $\widehat{\Theta}^r$.

We evaluate performance under a fixed-spike model, in which the signal eigenvalues $(\theta_i)_{i=1}^r$ do not vary with $n$. We measure loss using matrix norms $L_{\star, 1}(\Theta,\widehat{\Theta})$,  $\star \in \{ F,O,N \}$, as earlier, and evaluate asymptotic loss
following the  ``shrinkage descriptor" approach.
\begin{lemma} Let $\eta_n$ denote a sequence of shrinkers, possibly varying with $n$.
Under the fixed-spike model, suppose that the sequences of shrinker outputs converge:
\begin{align*}
 \eta_n(\lambda_i) & \xrightarrow{a.s.} \overline\eta_i \, , \hspace{-2cm}& 1 \leq i \leq r_+ \, , \\
\eta_n(\lambda_{n-(r-i)}) & \xrightarrow{a.s.} \overline\eta_i \, , \hspace{-2cm}  & r_+ < i \leq r \, .
\end{align*}
As before, we call the limits $(\overline \eta_i)_{i=1}^r$ the asymptotic shrinkage descriptors. Each loss $L_{\star,1}$ converges almost surely to a deterministic limit:
\[ \phantom{\,.} L_{\star,1}(\Theta, \widehat \Theta_{\eta_n}) \xrightarrow{a.s.} \overline{\cal L}_{\star}((\theta_i)_{i=1}^r, (\overline{\eta}_i)_{i=1}^r) \, .
\]
The asymptotic loss is sum- or max- decomposable into $r$ terms involving matrix norms applied to pivots of the
$2 \times 2$ matrices $\widetilde{A}$ and $\widetilde{B}$ introduced earlier. With $\theta_i$ denoting a spike parameter, $\overline{c}(\theta_i)$ the limiting cosine in (\ref{wig_cos}), and $\overline{s}^2(\theta_i) = 1 - \overline c^2(\theta_i)$, the decompositions are
 \begin{align*} \phantom{\,,} \overline{\cal L}_{F}((\theta_i)_{i=1}^r, (\overline{\eta}_i)_{i=1}^r) & = \bigg( \sum_{i=1}^r \big[ L_{F, 1}\big(\widetilde{A}(\theta_i),\widetilde{B}(\overline{\eta}_i,\overline{c}(\theta_i))\big) \big]^2 \bigg)^{1/2} \,, \\
 \overline{\cal L}_{O}((\theta_i)_{i=1}^r, (\overline{\eta}_i)_{i=1}^r) & = \max_{1 \leq i \leq r} L_{O,1} \big (\widetilde{A}(\theta_i),\widetilde{B}(\overline{\eta}_i,\overline{c}(\theta_i))\big) \, , \\
 \overline{\cal L}_{N}((\theta_i)_{i=1}^r,(\overline{\eta}_i)_{i=1}^r) & = \sum_{i=1}^r L_{N,1} \big(\widetilde{A}(\theta_i),\widetilde{B}(\overline{\eta}_i,\overline{c}(\theta_i))\big) \, .
\end{align*}
 \end{lemma}

Proceeding as before, we obtain closed forms of formally optimal shrinkers and losses, explicit in terms of $\theta$. As in previous sections, asymptotically optimal shrinkers on observables are constructed using the partial inverse $\theta(\lambda)$ (\ref{eq:theta-def}).

\begin{lemma}
\label{lem-wig-opt}
Formally optimal shrinkers and corresponding losses are given by
\begin{align}
    & \overline{\eta}^*(\theta|F) = \textup{sign}(\theta) (|\theta| - 1/|\theta|)_+ \,,&&   [\overline{\cal L}_{F}^1(\theta)]^2 =
   \begin{dcases}      \theta^2 (1 - 1/\theta^4) & |\theta| > 1 \\
               \theta^2 & 0  \leq |\theta|  \leq 1
               \end{dcases} \,      , \nonumber  \\
    & \overline{\eta}^*(\theta|O) = \theta \cdot 1_{\{|\theta| > 1 \}}\, ,
               &&\overline{\cal L}_{O}^1(\theta) =\begin{dcases}
               1 & |\theta| > 1\\
               |\theta| & 0 < |\theta| \leq 1
               \end{dcases} \, ,\label{eq:optloss-wig-oper}\\
    & \overline{\eta}^*(\theta|N) = \textup{sign}(\theta) \big(|\theta|-2 /|\theta|\big)_+ \, ,  && \overline{\cal L}_{N}^1(\theta) =
      \begin{dcases}
               2\sqrt{1-1/|\theta|^2} & |\theta| > \sqrt{2} \\
               |\theta| & 0 < |\theta| \leq \sqrt{2}
               \end{dcases} \,  . \nonumber
    \end{align}
\end{lemma}

Evidently, these expressions bear a strong formal resemblance to those we found earlier
for covariance shrinkage as $\gamma_n \rightarrow 0$: for $x > 0$,
\begin{align*}
   & \overline{\lambda}(x) = \tlam(x) \, , & \overline{c}(x) = \overset{\leftharpoonup}{c}(x) \, , \\
   & \overline{\eta}^*(x|\star) = \overset{\leftharpoonup}{\eta}^*(x | \star) \, , &  \overline{\cal L}_{\star}^1(x) = \lharp {\cal L}_{\star}^1(x) \,.
\end{align*}
Such similarities extend to hard thresholding; namely, the $L_{\star,1}$-optimal thresholds  $\overline{\tau}(\star)$  for the spiked Wigner model (to which eigenvalue magnitudes are compared to)
are equal to their counterparts in the $\gamma_n \rightarrow 0$ setting:
\[
\phantom{\,.}  \overline{\tau}(\star) = \lharpoonu{\tau}(\star) \, , \qquad \star \in \{F,O,N\} \, .
\]
These are not chance similarities. The empirical spectral distribution of $\gamma_n^{-1/2}(S - I)$ converges as $\gamma_n \rightarrow 0$ to the semicircle law (Bai and Yin \cite{BY88}). Spiked covariance formulas as $\gamma_n \rightarrow 0$ for eigenvalue inflation and eigenvector rotation---functions of the limiting spectral distribution---are therefore equivalent to those under the spiked Wigner model. By Lemmas \ref{lim-spg-spike-cosine} and \ref{lem-wig-opt}, this mandates identical shrinkage. In all essential quantitative aspects---eigenvalue inflation, eigenvector rotation, and optimal shrinkers and losses---the $\gamma_n \rightarrow 0$ covariance estimation and spiked Wigner settings are ``isomorphic.''

\section{Bidirectional Spiked Covariance Model}

Thus far we have discussed the spike covariance model assuming the
spiked eigenvalues deviate are {\it elevated}, $\ell_i > 1$. We now consider the possibility of {\it depressed} values, $\ell_i < 1$. Earlier results as $\gamma_n \rightarrow 0$ quickly adapt to this setting. We adopt the more general spike covariance model
 $\ell_{i} = \ell_{i,n} =  1 +  \overset{\leftharpoonup}{\ell}_i (1+o(1)) \sqrt{\gamma_n}$ where $\overset{\leftharpoonup}{\ell}_i$ may now
be either positive or negative. In the interest of brevity, results in this section are stated informally without proof.

Normalized bulk edges lie at $\pm 2$, and phase transitions of normalized eigenvalues occur at $\pm 1$. The ``bidirectional'' eigenvalue mapping function, $\tlam^\pm(\overset{\leftharpoonup}{\ell})$, is the odd extension of the ``unidirectional'' mapping (previously denoted by $\tlam$,
now by $\tlam^+$ for clarity):
\[ \phantom{\,.}
\tlam^\pm(\overset{\leftharpoonup}{\ell}) = \mbox{sign}(\overset{\leftharpoonup}{\ell}) \cdot \tlam^+( | \overset{\leftharpoonup}{\ell} |) \, ,
\]
while the cosine function $\lharp c^\pm(\overset{\leftharpoonup}{\ell}) = (1 - |\overset{\leftharpoonup}{\ell}|^{-2})_+$ is
the even extension of $\lharp c^+$ (previously denoted by $\lharp c$).

The connection to the spiked Wigner model is now even more apparent, eigenvalue mappings and cosine functions are identical:
\[
\tlam^\pm(\overset{\leftharpoonup}{\ell}) = \overline{\lambda}(\overset{\leftharpoonup}{\ell})\, , \hspace{2cm}  \overset{\leftharpoonup}{c}^\pm(\overset{\leftharpoonup}{\ell}) = \overline{c}(\overset{\leftharpoonup}{\ell}) \, .
\]
For Frobenius norm loss,
we have the ``bidirectionally optimal'' shrinker
\[
\phantom{\,.}   \overset{\leftharpoonup}{\eta}^\pm(\overset{\leftharpoonup}{\ell}|F) = \overline{\eta}(\overset{\leftharpoonup}{\ell}|F)  = \mbox{sign}(\overset{\leftharpoonup}{\ell}) \cdot ( |\overset{\leftharpoonup}{\ell}| - 1/|\overset{\leftharpoonup}{\ell}|)_+ \,.
\]
the odd extension of the ``unidirectionally optimal" shrinker,
while the optimal (rank-one) loss is $\sqrt{2 - 1/|\overset{\leftharpoonup}{\ell}|^2}$ for $|\overset{\leftharpoonup}{\ell}| > 1$ and $|\overset{\leftharpoonup}{\ell}|$ otherwise, the even extension of $\lharp {\cal L}_F^1$. Similarly, bidirectionally optimal shrinkers and corresponding losses under operator and nuclear norm losses are respectively the odd and even extensions of functions in Lemma \ref{lem-shr-opt}.

\section{Conclusion}

Although proportional-limit analysis has become popular in
recent years, many datasets---perhaps most---have many more rows
than columns or many more columns than rows.
We have studied eigenvalue shrinkage in each of these disproportional regimes and identified optimal
procedures under each, exhibiting closed-form expressions
for asymptotically optimal shrinkage functions and corresponding losses. We further identified a single closed-form nonlinearity for each loss function considered
which  can be  ``universally" applied across the proportional fixed-spiked or either disproportional varying-spike limit. Equivalent optimal shrinkage rules independently arise for matrix recovery under the spiked
Wigner  model.

\section*{Acknowledgements}

This work was supported by NSF DMS grant 1811614.