Bootstrap in High Dimension with Low Computation

Abstract: The bootstrap is a popular data-driven method to quantify statistical uncertainty, but for modern high-dimensional problems, it could suffer from huge computational costs due to the need to repeatedly generate resamples and refit models. We study the use of bootstraps in high-dimensional environments with a small number of resamples. In particular, we show that with a recent "cheap" bootstrap perspective, using a number of resamples as small as one could attain valid coverage even when the dimension grows closely with the sample size, thus strongly supporting the implementability of the bootstrap for large-scale problems. We validate our theoretical results and compare the performance of our approach with other benchmarks via a range of experiments.

\section{Introduction\label{sec:introduction}}

The bootstrap is a widely used method for statistical inference, notably confidence interval construction and uncertainty quantification (e.g. \cite{efron1994introduction,davison1997bootstrap,shao2012jackknife,hall1988bootstrap}). Its main idea is to resample data and use the distribution of resample estimates to approximate a sampling distribution. Typically, this approximation requires running many Monte Carlo replications to generate the resamples and refit models. This is affordable for classical small-scale problems, but for modern large-scale problems this repeated fitting could impose tremendous computation concerns. This issue motivates an array of recent works to curb the computation effort, mostly through a ``subsampling'' perspective that fits models on smaller data sets in the bootstrap process, e.g., \cite{kleiner2012big,lu2020uncertainty,giordano2019swiss,schulam2019can,alaa2020discriminative}.

In contrast to subsampling, we consider in this paper the reduction in bootstrap computation cost by using fewer number of Monte Carlo replications or resamples. In particular, a recent initial work \cite{lam2022cheap} has shown that, to construct statistically valid confidence intervals, it is possible to reduce the resampling effort dramatically from standard bootstraps, even down to one Monte Carlo replication. The rough idea of this approach, which \cite{lam2022cheap} called a ``cheap'' bootstrap, is to exploit the approximate independence among the resample estimates and original estimate instead of only the distributional closeness. This argument is asymptotic that unfortunately lacks finite-sample and dimensionality consideration. The latter is important for applications in modern machine learning and computational simulation problems, because it is precisely the scale of these problems that necessitates the saving of Monte Carlo costs.

Our main contribution in this paper is to develop finite-sample analyses for the cheap bootstrap procedure. Our results are three-fold. First, we present general bounds on the aggregation of $B$ resample estimates, where $B$ is a low number, that allow us to compare the cheap bootstrap coverage errors with more traditional quantile-based bootstrap methods including the basic and percentile bootstraps (e.g., \cite{davison1997bootstrap} Section 5.2-5.3). These latter approaches require a large amount of resamples to approximate the sampling distribution. Our analyses show that the cheap bootstrap using as low as one resample achieves similar coverage error bounds as these approaches using infinite resamples, thus suggesting the capability to dramatically reduce computation cost.

Second, we specialize our bounds to the function-of-mean model that is customary in the high-dimensional Berry-Esseen and central limit theorem (CLT) literature (\cite{pinelis2016optimal,zhilova2020nonclassical}). We derive bounds explicit on problem parameters, including the dimension $p$, the sample size $n$, and the number of resamples $B$. These bounds reveal vanishing coverage errors for the cheap bootstrap when $p=o(n)$, for any given $B\geq1$. Note that the function-of-mean model does not capture all interesting problems, but it has been commonly used -- and in fact, appears the only model used in deriving finite-sample CLT errors for technicality reasons. Our bounds shed light that, at least for this wide class of models, using a small number of resamples achieves a good coverage even in a dimension $p$ growing closely with $n$. Third, we also specialize our bounds to linear functions with weaker tail conditions, which have orders independent of $p$ under certain conditions on the $L_p$ norm or Orlicz norm of the linearly scaled random variable.

In addition to theoretical bounds, we test the cheap bootstrap and compare with other benchmarks on large-scale problems, including high-dimensional linear regression, high-dimensional logistic regression, computational simulation modeling, and a real-world data set RCV1-v2 (\cite{lewis2004rcv1}). To give a sense of the comparisons, we briefly illustrate here the general conclusion observed in our experiments: Figure \ref{figure_linear_vs_B} (a) shows the coverage probabilities of $95\%$-level confidence intervals for three regression coefficients with corresponding true values $0,2,-1$ in a 9000-dimensional linear regression (in Section \ref{sec:num}). We see the cheap bootstrap coverage probabilities are close to the nominal level $95\%$ even with one resample, but the basic and percentile bootstraps only attain around $80\%$ coverage with ten resamples. In this example, one Monte Carlo replication to obtain each resample estimate takes around 4 minutes in the virtual machine e2-highmem-2 in Google Cloud Platform. Therefore, the cheap bootstrap only requires 4 minutes to obtain a statistically valid interval, but the standard bootstrap methods are still far from the nominal coverage even after more than a 40-minute run. Figure \ref{figure_linear_vs_B} (b) shows the average interval widths. This reveals the price of a wider interval for the cheap bootstrap when the Monte Carlo budget is very small, but considering the low coverages in the other two methods and the fast decay of the cheap bootstrap width for the first few number of resamples, such a price appears secondary.

\begin{figure}[htbp]
\centering

\subfloat[Coverage probability]{\includegraphics[width=0.45\textwidth]{figures/linear_beta_coverage.pdf}}
\subfloat[Confidence interval width]{\includegraphics[width=0.45\textwidth]{figures/linear_beta_width.pdf}}

\caption{Empirical coverage probabilities and confidence interval widths for different number of resamples in a linear regression}%
\label{figure_linear_vs_B}

\end{figure}

\textit{Notation.} For a random vector $X$, we write $X^{k}$ as the tensor power $X^{\otimes k}$. The vector norm is taken as the usual Euclidean norm. The matrix and tensor norm is taken as the operator norm. For a square matrix $M$, $tr(M)$ denotes the trace of $M$. $I_{p\times p}$ denotes the identity matrix in $\mathbb{R}^{p\times p}$ and $\mathbf{1}_{p}$ denotes the vector in $\mathbb{R}^{p}$ whose components are all $1$.
$\Phi$ denotes the cumulative distribution function of the standard normal. $C^2(\mathbb{R}^p)$ denotes the twice continuously differentiable function on $\mathbb{R}^p$. Throughout the whole paper, we use $C>0$ (without subscripts) to denote a universal constant which may vary each time it appears. We use $C_{1},C_{2},\ldots$ to denote constants that could depend on other parameters and we will clarify their dependence when using them.

\section{Background on Cheap Bootstrap\label{sec:method}}

We briefly review the cheap bootstrap method proposed in \cite{lam2022cheap}. Suppose we are interested in estimating a target statistical quantity $\psi:=\psi(P_{X})$ where $\psi(\cdot):\mathcal{P}\mapsto\mathbb{R}$ is a functional defined on the probability measure space $\mathcal{P}$. Given i.i.d. data $X_{1},\ldots,X_{n}\in\mathbb{R}^{p}$ following the unknown distribution $P_{X}$, we denote the empirical distribution as $\hat{P}_{X,n}(\cdot):=(1/n)\sum_{i=1}^nI(X_i\in\cdot)$. A natural point estimator is $\hat{\psi}_{n}:=\psi(\hat{P}_{X,n})$.

The cheap bootstrap confidence interval for $\psi$ is constructed as follows. Conditional on $X_{1},\ldots,X_{n}$, we independently resample, i.e., sample with replacement, the data for $B$ times to obtain resamples $\{X_{1}^{\ast b},\ldots,X_{n}^{\ast b}\},b=1,\ldots,B$. Denoting $\hat{P}_{X,n}^{\ast b}$ as the resample empirical distributions, we construct $B$ resample estimates $\hat\psi_{n}^{\ast b}:=\psi(\hat{P}_{X,n}^{\ast b})$. A $(1-\alpha)$-level confidence interval is then given by
\begin{equation}
\left[\hat{\psi}_{n}-t_{B,1-\alpha/2}S_{n,B},\hat{\psi}_{n}+t_{B,1-\alpha/2}S_{n,B}\right],\label{cheap CI}
\end{equation}
where $S_{n,B}^{2}=(1/B)\sum_{b=1}^{B}(\hat\psi_{n}^{\ast b}-\hat{\psi}_{n})^{2}$, and $t_{B,1-\alpha/2}$ is the $(1-\alpha/2)$-th quantile of $t_B$, the $t$-distribution with degree of freedom $B$.
The quantity $S_{n,B}^2$ resembles the sample variance of the resample estimates $\hat\psi_n^{*b}$'s, in the sense that as $B\to\infty$, $S_{n,B}^2$ approaches the bootstrap variance $Var_*(\hat\psi_n^*)$ (where $Var_*(\cdot)$ denotes the variance of a resample conditional on the data). In this way, \eqref{cheap CI} reduces to the normality interval with a ``plug-in'' estimator of the standard error term when $B$ and $n$ are both large. However, the key here is that \emph{$B$ does not need to be large}. In fact, \cite{lam2022cheap} Theorem 1 shows that, under conditions on par with standard bootstrap methods, \eqref{cheap CI} is an asymptotically exact $(1-\alpha)$-level confidence interval \emph{for any fixed $B\geq1$}, i.e.,
\begin{equation}
P(\psi\in[\hat{\psi}_{n}-t_{B,1-\alpha/2}S_{n,B},\hat{\psi}_{n}+t_{B,1-\alpha/2}S_{n,B}])\to1-\alpha, \text{\ \ as\ \ }n\to\infty\label{asymptotic exact}
\end{equation}
where $P$ is the probability with respect to both the data and the randomness in the resampling process.

We briefly explain the asymptotic argument that gives rise to \eqref{asymptotic exact}, and along the way we contrast with standard bootstrap methods. Under suitable conditions, the sampling distribution of an estimate $\hat\psi_n$ and the distribution of a resample estimate $\hat\psi_n^*$ are approximately equal. More formally, they are equal in the asymptotic sense of two CLTs $\sqrt{n}(\hat{\psi}_{n}-\psi)\overset{d}{\rightarrow}N(0,\sigma^{2})$ for some $\sigma^{2}>0$, and $\sqrt{n}(\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n})\overset{d}{\rightarrow}N(0,\sigma^{2})$  (conditional on $X_{1},\ldots,X_{n}$ in probability) for the same $\sigma^{2}$.
Note that the distribution of $\hat{\psi}_{n}-\psi$ allows us to construct a confidence interval for $\psi$. However, as this distribution is typically unknown, the standard bootstrap idea approximates it by the distribution of $\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n}$ which can be readily simulated. Suppose we use the $\alpha/2$ and $(1-\alpha/2)$-th quantiles of $\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n}$, called $q_{\alpha/2}$ and $q_{1-\alpha/2}$, to construct $[\hat{\psi}_{n}-q_{1-\alpha/2},\hat{\psi}_{n}-q_{\alpha/2}]$ as a $(1-\alpha)$-level confidence interval. Then this is known as the basic bootstrap (\cite{davison1997bootstrap} Section 5.2). Alternatively, suppose we obtain the $\alpha/2$ and $(1-\alpha/2)$-th quantiles of $\hat{\psi}_{n}^{\ast}$, say $q_{\alpha/2}$ and $q_{1-\alpha/2}$ again, and use $[q_{\alpha/2},q_{1-\alpha/2}]$. Then this becomes the percentile bootstrap (\cite{davison1997bootstrap} Section 5.3), whose simplicity is justified by further exploiting the symmetry of the asymptotic normal distribution.

On the other hand, the cheap bootstrap interval \eqref{cheap CI} utilizes the following joint convergence%
\begin{equation}
\sqrt{n}(\hat{\psi}_{n}-\psi,\hat\psi_{n}^{\ast1}-\hat{\psi}_{n},\ldots,\hat\psi_{n}^{\ast B}-\hat{\psi}_{n})\overset{d}{\rightarrow}(\sigma Z_{0},\sigma Z_{1},\ldots,\sigma Z_{B}), \text{\ \ as\ \ }n\to\infty \label{joint_convergence}%
\end{equation}
where $Z_{b},b=0,\ldots,B$ are i.i.d. standard normal, a result can be derived by combining the two aforementioned CLTs. From \eqref{joint_convergence}, we can establish the convergence of a pivotal $t$-statistic $(\hat\psi_n-\psi)/S_{n,B}\overset{d}{\rightarrow}t_B$
which gives \eqref{asymptotic exact}. The above shows that, with $B$ fixed as small as $1$, \eqref{cheap CI} already offers a coverage close to the nominal level as $n\to\infty$. In this argument, the approximation accuracy of $(\hat\psi_n-\psi)/S_{n,B}$ by the $t_B$ random variable is crucial and will be the main focus in our finite-sample analysis.

\section{Finite-Sample Coverage Error Bounds\label{sec:main}}
We describe our finite-sample results at three levels, first under general assumptions (Section \ref{sec:general bounds}), then more explicit bounds under the function-of-mean model and sub-Gaussianity of $X$ (Section \ref{sec:nonlinear}), finally bounds for a linear function under weaker tail assumptions on $X$ (Section \ref{sec:linear}).

\subsection{General Finite-Sample Bounds\label{sec:general bounds}}

We have the following finite-sample bound for the cheap bootstrap:

\begin{theorem}
\label{general_CB_coverage}Suppose we have the finite-sample accuracy for the estimator $\hat{\psi}_{n}$
\begin{equation}
\sup_{x\in\mathbb{R}}\left\vert P(\sqrt{n}(\hat{\psi}_{n}-\psi)\leq x)-\Phi(x/\sigma)  \right\vert \leq\mathcal{E}_{1},\label{general_Berry_Esseen}%
\end{equation}
and with probability at least $1-\beta$ we have the finite-sample accuracy for the bootstrap estimator $\hat{\psi}_{n}^{\ast}$%
\begin{equation}
\sup_{x\in\mathbb{R}}\left\vert P^{\ast}(\sqrt{n}(\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n})\leq x)-\Phi(x/\sigma) \right\vert\leq\mathcal{E}_{2},\label{general_bootstrap_CLT}%
\end{equation}
where $\sigma>0$, $\mathcal{E}_{1}$ and $\mathcal{E}_{2}$ are deterministic quantities, and $P^*$ denotes the probability on a resample conditional on the data. Then the cheap bootstrap coverage error satisfies, for any $B\geq1$,%
\begin{equation}
\left\vert P(\psi\in\lbrack\hat{\psi}_{n}-t_{B,1-\alpha/2}S_{n,B},\hat{\psi}_{n}+t_{B,1-\alpha/2}S_{n,B}])-(1-\alpha)\right\vert \leq 2\mathcal{E}_{1}+2B\mathcal{E}_{2}+\beta.\label{error CB}
\end{equation}
\end{theorem}

Condition (\ref{general_Berry_Esseen}) is a Berry-Esseen bound (\cite{bentkus2003dependence,pinelis2016optimal}) that gauges the normal approximation for the original estimate $\hat\psi_n$. Condition (\ref{general_bootstrap_CLT}) manifests a similar normal approximation for the resample estimate $\hat\psi_n^*$, and has been a focus in the high-dimensional CLT literature (\cite{zhilova2020nonclassical,lopes2020central,chernozhukov2020nearly}). Both conditions (in their asymptotic form) are commonly used to establish the validity of standard bootstrap methods. Theorem \ref{general_CB_coverage} shows that, under these conditions, the coverage error of the cheap bootstrap with any $B\geq1$ can be controlled. Note that Theorem \ref{general_CB_coverage} is very general in the sense that there is no direct assumption applied on the form of $\psi(\cdot)$ -- All we assume is approximate normality in the sense of (\ref{general_Berry_Esseen}) and (\ref{general_bootstrap_CLT}). Due to technical delicacies, in the bootstrap literature, finite-sample or higher-order coverage errors are typically obtainable only with specific models (\cite{hall2013bootstrap,zhilova2020nonclassical,lopes2020central,chernozhukov2020nearly}), the most popular being the function-of-mean model (see Section \ref{sec:nonlinear}) or even simply the sample mean. In contrast, the bound in Theorem \ref{general_CB_coverage} that concludes the sufficiency in using a very small $B$ is a general statement that does not depend on the delicacies of $\psi(\cdot)$. The detailed proof of Theorem \ref{general_CB_coverage} is in Appendix C (and so are the proofs of all other theorems). Below we give a sketch of the main argument.

\textbf{Proof sketch of Theorem \ref{general_CB_coverage}.} \emph{Step 1: }We write the coverage probability as the expected value (with respect to data) of a multiple integral with respect to the distributions of $\sqrt{n}(\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n})$ (denoted by $Q^{\ast}$, conditional on data), i.e.,%
\begin{align}
&  P(\psi\in\lbrack\hat{\psi}_{n}-t_{B,1-\alpha/2}S_{n,B},\hat{\psi}_{n}+t_{B,1-\alpha/2}S_{n,B}])\nonumber\\
&  =P\left(  |\sqrt{n}(\hat{\psi}_{n}-\psi)|\leq t_{B,1-\alpha/2}\sqrt{{\textstyle\sum\nolimits_{b=1}^{B}}(\sqrt{n}(\hat{\psi}_{n}^{\ast b}-\hat{\psi}_{n}))^{2}/B}\right)  \nonumber\\
&  =E\left[  \int\cdots\int_{|\sqrt{n}(\hat{\psi}_{n}-\psi)|\leq t_{B,1-\alpha/2}\sqrt{\sum_{b=1}^{B}z_{b}^{2}/B}}dQ^{\ast}(z_{B})\cdots dQ^{\ast}(z_{1})\right]  .\label{multiple_integral}%
\end{align}

\emph{Step 2: }Suppose (\ref{general_bootstrap_CLT}) happens and denote this event by $\mathcal{A}$ which satisfies $P(\mathcal{A}^{c})\leq\beta$. For each $b=1,\ldots,B$, given all other $z_{b^{\prime}},b^{\prime}\neq b$, the integration region is of the form $z_{b}\in(-\infty,-q]\cup\lbrack q,\infty)$ for some $q\geq0$. Then we can replace the distribution $Q^{\ast}$ by the distribution of $N(0,\sigma^{2})$ (denoted by $P_{0}$) with controlled error given in (\ref{general_bootstrap_CLT}) and obtain%
\begin{align}
&  E\left[  \int\cdots\int_{|\sqrt{n}(\hat{\psi}_{n}-\psi)|\leq t_{B,1-\alpha/2}\sqrt{\sum_{b=1}^{B}z_{b}^{2}/B}}dQ^{\ast}(z_{B})\cdots dQ^{\ast}(z_{1})\right]  \nonumber\\
&  =E\left[  \int\cdots\int_{|\sqrt{n}(\hat{\psi}_{n}-\psi)|\leq t_{B,1-\alpha/2}\sqrt{\sum_{b=1}^{B}z_{b}^{2}/B}}dP_{0}(z_{B})\cdots dP_{0}(z_{1})\right]  +R_{1},\label{sketch_expansion1}%
\end{align}
where $|R_{1}|\leq2B\mathcal{E}_{2}+\beta$ accounts for the error from (\ref{general_bootstrap_CLT}) and the small probability event $\mathcal{A}^{c}$.

\emph{Step 3: }Following the same logic in Step 2 and noticing that the integration region for $\sqrt{n}(\hat{\psi}_{n}-\psi)$ is $[-q,q]$ for some $q\geq0$, we can also replace the distribution of $\sqrt{n}(\hat{\psi}_{n}-\psi)$ by the distribution $P_{0}$ with controlled error $|R_{2}|\leq2\mathcal{E}_{1}$ according to (\ref{general_Berry_Esseen}):%
\begin{align}
&  E\left[  \int\cdots\int_{|\sqrt{n}(\hat{\psi}_{n}-\psi)|\leq t_{B,1-\alpha/2}\sqrt{\sum_{b=1}^{B}z_{b}^{2}/B}}dP_{0}(z_{B})\cdots dP_{0}(z_{1})\right]  \nonumber\\
&  =\int\int\cdots\int_{|z_{0}|\leq t_{B,1-\alpha/2}\sqrt{\sum_{b=1}^{B}z_{b}^{2}/B}}dP_{0}(z_{B})\cdots dP_{0}(z_{1})dP_{0}(z_{0})+R_{2}=1-\alpha+R_{2}.\label{sketch_expansion2}%
\end{align}

\emph{Step 4: }Plugging (\ref{sketch_expansion1}) and (\ref{sketch_expansion2}) back into (\ref{multiple_integral}), we can express the coverage probability as a sum of the nominal level and the remainder term:%
\[
P(\psi\in\lbrack\hat{\psi}_{n}-t_{B,1-\alpha/2}S_{n,B},\hat{\psi}_{n}+t_{B,1-\alpha/2}S_{n,B}])=1-\alpha+R_{1}+R_{2}%
\]
with error $|R_{1}+R_{2}|\leq2\mathcal{E}_{1}+2B\mathcal{E}_{2}+\beta$. This gives our conclusion.\hfill\qed

Theorem \ref{general_CB_coverage} is designed to work well for small $B$ (our target scenario), but deteriorates when $B$ grows. However, in the latter case, we can strengthen the bound to cover the large-$B$ regime with additional conditions on the variance estimator (see Appendix A).

We compare with standard basic and percentile bootstraps using $B=\infty$. Below is a generalization of \cite{zhilova2020nonclassical} which focuses only on the basic bootstrap under the function-of-mean model.

\begin{theorem}
\label{general_other_coverages}Suppose the conditions in Theorem \ref{general_CB_coverage} hold. If $q_{\alpha/2}$, $q_{1-\alpha/2}$ are the $\alpha/2$-th and $(1-\alpha/2)$-th quantiles of $\hat{\psi}_{n}^{\ast}-\hat{\psi}_{n}$ respectively given $X_{1},\ldots,X_{n}$, then a finite-sample bound on the basic bootstrap coverage error is given by%
\begin{equation}
|P(\hat{\psi}_{n}-q_{1-\alpha/2}\leq\psi\leq\hat{\psi}_{n}-q_{\alpha/2})-(1-\alpha)|\leq2\mathcal{E}_{1}+2\mathcal{E}_{2}+2\beta.\label{error basic}
\end{equation}
If $q_{\alpha/2}$, $q_{1-\alpha/2}$ are the $\alpha/2$-th and $(1-\alpha/2)$-th quantiles of $\hat{\psi}_{n}^{\ast}$ respectively given $X_{1},\ldots,X_{n}$, then a finite-sample bound on the percentile bootstrap coverage error is given by%
\begin{equation}
|P(q_{\alpha/2}\leq\psi\leq q_{1-\alpha/2})-(1-\alpha)|\leq2\mathcal{E}_{1}+2\mathcal{E}_{2}+2\beta.\label{error percentile}
\end{equation}

\end{theorem}

In view of Theorems \ref{general_CB_coverage} and \ref{general_other_coverages}, the cheap bootstrap with any fixed $B$ can achieve the same order of coverage error bound as the basic and percentile bootstraps with $B=\infty$, in the sense that
\begin{equation}
(1/2)\ \text{EB}_\text{Quantile}\leq\text{EB}_\text{Cheap}\leq B\ \text{EB}_\text{Quantile},\label{bound comparison}
\end{equation}
where $\text{EB}_\text{Cheap}$ is the RHS error bound of \eqref{error CB} and $\text{EB}_\text{Quantile}$ is that of \eqref{error basic} or \eqref{error percentile}. This shows that, to attain a good coverage that is on par with standard basic/percentile bootstraps, it suffices to use the cheap bootstrap with a small $B$ which could save computation dramatically.

Besides coverage, another important quality of confidence interval is its width. To this end, note that for any fixed $B$, (\ref{general_bootstrap_CLT}) ensures that $\sqrt{n}S_{n,B}\Rightarrow\sigma\sqrt{\chi_{B}^{2}/B}$ (unconditionally as $n\rightarrow\infty$ with proper  model assumptions) . Therefore, the half-width of (\ref{cheap CI}) is approximately $t_{B,1-\alpha/2}\sigma\sqrt{\chi_{B}^{2}/(nB)}$ with expected value
\begin{equation}
E\left[  t_{B,1-\alpha/2}\sigma\sqrt{\frac{\chi_{B}^{2}}{nB}}\right]=t_{B,1-\alpha/2}\sigma\sqrt{\frac{2}{Bn}}\frac{\Gamma((B+1)/2)}{\Gamma(B/2)},\label{half_width}
\end{equation}
where $\Gamma(\cdot)$ is the gamma function. Since the dimensional impact is hidden in $\sigma$ which is a common factor in the expected width as $B$ varies, we can see $p$ does not affect the relative width behavior as $B$ changes. In particular, from \eqref{half_width} we can readily see that the inflation of the expected width relative to the case $B=\infty$ is 417.3\% for $B=1$, and dramatically reduces to 94.6\%, 24.8\% and 10.9\% for $B=2,5,10$, thus giving an interval with both correct coverage and short width using a small computation budget $B$.

In the next sections, we will apply Theorem \ref{general_CB_coverage} to obtain explicit bounds for specific high-dimensional models. Here, in relation to \eqref{bound comparison}, we briefly comment that the order of the coverage error bounds for these models is of order $1/\sqrt n$, both for the cheap bootstrap (which we will derive) and state-of-the-art high-dimensional bootstrap. This is in contrast to the typical $1/n$ coverage error in two-sided bootstrap confidence intervals in low dimension (see \cite{hall2013bootstrap} Section 3.5 for quantile-based bootstraps and \cite{lam2022cheap} Section 3.2 for cheap bootstrap).

\subsection{Function-Of-Mean Models \label{sec:nonlinear}}

We now specialize to the function-of-mean model $\psi=g(\mu)$ for a mean vector $\mu=E[X]\in\mathbb{R}^{p}$ and function $g:\mathbb{R}^{p}\mapsto\mathbb{R}$, which allows us to construct more explicit bounds. The original estimate $\hat{\psi}_{n}$ and resample estimate $\hat\psi_{n}^{\ast b}$ are now given by $g(\bar{X}_{n})$ and $g(\bar{X}_{n}^{\ast b})$ respectively, where $\bar X_n$ denotes the sample mean of data and $\bar X_n^{*b}$ denotes the resample mean of $X_1^{*b},\ldots,X_n^{*b}$. We assume:
\begin{assumption}\label{smoothness_assumption}
The function $g(x)\in C^2(\mathbb{R}^p)$ has Hessian matrix $H_{g}(x)$ with uniformly bounded eigenvalues, that is, $\exists$ a constant $C_{H_{g}}>0$ s.t. $\sup_{x\in\mathbb{R}^{p}}|a^{\top}H_{g}(x)a|\leq C_{H_{g}}||a||^{2},\forall a\in\mathbb{R}^{p}$.
\end{assumption}
\begin{assumption}\label{nonlinear_assumption}
$X$ is sub-Gaussian, i.e., there is a constant $\tau^{2}>0$ s.t. $E[\exp(a^{\top}(X-\mu))]\leq\exp(||a||^{2}\tau^{2}/2),\forall a\in \mathbb{R}^{p}$. %
Furthermore, $X$ has a density bounded by a constant $C_{X}$ and its covariance matrix $\Sigma$ is positive definite with the smallest eigenvalue $\lambda_{\Sigma}>0$.
\end{assumption}

Based on Theorem \ref{general_CB_coverage}, we derive the following explicit bound:

\begin{theorem}
\label{CB_coverage_nonlinear}Suppose the function $g$ satisfies Assumption \ref{smoothness_assumption} and random vector $X$ satisfies Assumption \ref{nonlinear_assumption}. Moreover, assume $||\nabla g(\mu)||>C_{\nabla g}\sqrt{p}$ for some constant $C_{\nabla g}>0$. Then we have
\begin{align*}
& \left\vert P(g(\mu)\in[g(\bar{X}_{n})-t_{B,1-\alpha/2}S_{n,B},g(\bar{X}_{n})+t_{B,1-\alpha/2}S_{n,B}])  -(1-\alpha)\right\vert\\
&  \leq \frac{6}{n}+BC\left(\frac{m_{31}}{\sqrt{n}\sigma^{3}}+\frac{C_{H_{g}}m_{31}^{1/3}tr(\Sigma)}{\sqrt{n}\sigma^{2}}+\frac{C_{H_{g}}m_{32}^{2/3}}{n^{5/6}\sigma}+\frac{C_{H_{g}}m_{31}^{1/3}m_{32}^{2/3}}{n\sigma^{2}}\right. \\
&  +\frac{C_{H_{g}}\tau^{2}}{C_{\nabla g}\sqrt{\lambda_{\Sigma}}}\left(1+\frac{\log n}{p}\right)  \sqrt{\frac{p}{n}}+\frac{||E[(X-\mu)^{3}]||}{\lambda_{\Sigma}^{3/2}}\frac{1}{\sqrt{n}}+\frac{\tau^{3}}{\lambda_{\Sigma}^{3/2}}\left(  1+\frac{\log n}{p}\right)  ^{3/2}\frac{1}{\sqrt{n}}\\
&  \left.  +\frac{\tau^{4}\sqrt{p}}{\lambda_{\Sigma}^{2}n}\left(  1+\frac{\log n}{p}\right)  ^{1/2}+\frac{\tau^{2}\sqrt{p}}{\lambda_{\Sigma}n}\left(1+\frac{\log n}{p}\right)  ^{1/2}+\frac{\tau^{3}\sqrt{p}}{\lambda_{\Sigma}^{3/2}n}\left(  1+\frac{\log n}{p}\right)  \right) \\
&  +BC_{1}\left(  \frac{\tau^{4}(\log n)^{3/2}}{\lambda_{\Sigma}^{2}\sqrt{n}}+\frac{\tau^{2}(\log n)^{3/2}}{\lambda_{\Sigma}\sqrt{n}}+\frac{\tau^{3}}{\lambda_{\Sigma}^{3/2}\sqrt{n}}\left(  1+\frac{\log n}{p}\right)^{1/2}(\log n+\log p)\sqrt{\log n}\right)  ,
\end{align*}
where $m_{31}=E[|\nabla g(\mu)^{\top}(X-\mu)|^{3}]$, $m_{32}:=E[||X-\mu||^{3}]$, $\sigma^{2}=\nabla g(\mu)^{\top}\Sigma\nabla g(\mu)$, $C$ is a universal constant and $C_{1}$ is a constant only depending on $C_{X}$.
\end{theorem}

Theorem \ref{CB_coverage_nonlinear} is obtained by tracing the implicit quantities in Theorem \ref{general_CB_coverage} for the function-of-mean model, by extracting the dependence on problem parameters in the Berry-Esseen theorems for the multivariate delta method (\cite{pinelis2016optimal}) and the standard bootstrap (\cite{zhilova2020nonclassical}). The bound in Theorem \ref{CB_coverage_nonlinear} can be simplified with reasonable assumptions on the involved quantities:

\begin{corollary}\label{concise_CB_coverage_nonlinear}
Suppose the conditions in Theorem \ref{CB_coverage_nonlinear} hold. Moreover, suppose that $\tau,C_{H_{g}},C_{1}=O(1)$, $C_{\nabla g},\lambda_{\Sigma}=\Theta(1)$, $||\nabla g(\mu)||^{2}=O(p)$, $\sigma^{2}=\Theta(p)$ and $||E[(X-\mu)^{3}]||=O(1)$. Then as $p,n\rightarrow\infty$,
\begin{align*}
& \left\vert P(g(\mu)\in[g(\bar{X}_{n})-t_{B,1-\alpha/2}S_{n,B},g(\bar{X}_{n})+t_{B,1-\alpha/2}S_{n,B}])  -(1-\alpha)\right\vert\\
&  =B\times O\left(  \left(  1+\frac{\log n}{p}\right)  \sqrt{\frac{p}{n}}+\frac{1}{\sqrt{n}}\left(  1+\frac{\log n}{p}\right)  ^{1/2}(\log n+\log p)\sqrt{\log n}\right)  .
\end{align*}
\end{corollary}

In Corollary \ref{concise_CB_coverage_nonlinear}, the cheap bootstrap coverage error shrinks to $0$ as $n\rightarrow\infty$ if $p=o(n)$, i.e., the problem dimension grows slower than $n$ in any arbitrary fashion. Recall that $||E[(X-\mu)^{3}]||$ denotes the operator norm of the third order tensor $E[(X-\mu)^{3}]$, and so the assumption $||E[(X-\mu)^{3}]||=O(1)$ holds if the components of $X$ are independent (or slightly weakly dependent). Other order assumptions in Corollary \ref{concise_CB_coverage_nonlinear} are natural. An example of the function-of-mean model is $g(\mu)=||\mu||^2$, used also in \cite{zhilova2020nonclassical}, whose confidence interval becomes a simultaneous confidence region for the mean vector $\mu$.

\subsection{Linear Functions\label{sec:linear}}

We consider a further specialization to linear $g$ where, instead of sub-Gaussanity of $X$, we are now able to use weaker tail conditions. Assume $g(x)=g_{1}^{\top}x+g_{2}$, where $g_{1}\in\mathbb{R}^{p}$ and $g_{2}\in\mathbb{R}$ are known. Then $g(\bar{X}_{n})$ and $g(\bar{X}_{n}^{\ast b})$ are essentially the sample mean and resample mean of i.i.d. random variables $g_{1}^{\top}X_{i}+g_{2},i=1,\ldots,n$.

First, we consider the case where $g_{1}^{\top}(X-\mu)$ is sub-exponential, i.e.,
$||g_{1}^{\top}(X-\mu)||_{\psi_{1}}:=\inf\{\lambda>0:E[\psi_{1}(|g_{1}^{\top}(X-\mu)|/\lambda)]\leq1\}<\infty
$,
where $||\cdot||_{\psi_{1}}$ is the Orlicz norm induced by the function $\psi_{1}(x)=e^{x}-1$. Sub-exponential property is a weaker tail condition than sub-Gaussianity; see e.g. \cite{vershynin2018high} Sections 2.5 and 2.7. Under this condition, we have:
\begin{theorem}
\label{CB_coverage_subexp}Suppose $g$ is a linear function in the form $g(x)=g_{1}^{\top}x+g_{2}$. Assume that $\sigma^{2}=g_{1}^{\top}\Sigma g_{1}>0$ and $||g_{1}^{\top}(X-\mu)||_{\psi_{1}}<\infty$. Then for any $n\geq3$ and for a universal constant $C$, we have the following finite-sample bound on the cheap bootstrap coverage error%
\begin{align*}
& \left\vert P(g(\mu)\in[g(\bar{X}_{n})-t_{B,1-\alpha/2}S_{n,B},g(\bar{X}_{n})+t_{B,1-\alpha/2}S_{n,B}])  -(1-\alpha)\right\vert\\
&  \leq \frac{C}{n}+BC\left(  \frac{E[|g_{1}^{\top}(X-\mu)|^{3}]}{\sigma^{3}\sqrt{n}}+\frac{||g_{1}^{\top}(X-\mu)||_{\psi_{1}}^{4}\log^{11}(n)}{\sigma^{4}\sqrt{n}}\right)  +\frac{CE[|g_{1}^{\top}(X-\mu)|^{3}]}{\sigma^{3}\sqrt{n}}.
\end{align*}
\end{theorem}
Note that the bootstrap in Theorem \ref{CB_coverage_subexp} effectively applies on the univariate $g_{1}^{\top}(X-\mu)$. Nonetheless, proving Theorem \ref{CB_coverage_subexp} requires tools from high-dimensional CLT (\cite{lopes2020central,chernozhukov2020nearly}), as this turns out to be the only line of work that investigates finite-sample bootstrap errors (for mean estimation). The order of the bound in terms of $p$ is controlled by $g_{1}^{\top}(X-\mu)$, and so if the latter is well-scaled by its standard deviation $\sigma$ in the sense that $E[|g_{1}^{\top}(X-\mu)/\sigma|^{3}],||g_{1}^{\top}(X-\mu)/\sigma||_{\psi_{1}}=O(1)$ (e.g., $X$ follows a multivariate normal distribution), then the order is independent of $p$, which means the error goes to $0$ for any $p$ as long as $n\rightarrow\infty$. However, if the orders of $E[|g_{1}^{\top}(X-\mu)/\sigma|^{3}]$ and  $||g_{1}^{\top}(X-\mu)/\sigma||_{\psi_{1}}$ depend on $p$, then the growing rate of $p$ must be restricted by $n$ to ensure the error goes to $0$.

Next, we further weaken the tail condition on $g_{1}^{\top}(X-\mu)$. We only assume $E[|g_{1}^{\top}(X-\mu)|^{q}]<\infty$ for some $q\geq4$. In this case, we have the following:

\begin{theorem}
\label{CB_coverage_moments}Suppose $g$ is a linear function in the form of $g(x)=g_{1}^{\top}x+g_{2}$. Assume that $\sigma^{2}=g_{1}^{\top}\Sigma g_{1}>0$ and $E[|g_{1}^{\top}(X-\mu)|^{q}]<\infty$ for some $q\geq4$. Then for any $n\geq3$, we have the following finite-sample bound on the cheap bootstrap coverage accuracy%
\begin{align*}
& \left\vert P(g(\mu)\in[g(\bar{X}_{n})-t_{B,1-\alpha/2}S_{n,B},g(\bar{X}_{n})+t_{B,1-\alpha/2}S_{n,B}])  -(1-\alpha)\right\vert\\
&  \leq \frac{B  C_{1}\sqrt{\log n}}{n^{1/2-3/(2q)}}\max\left\{  E[|g_{1}^{\top}(X-\mu)/\sigma|^{q}]^{1/q},\sqrt{E[|g_{1}^{\top}(X-\mu)/\sigma|^{4}]}\right\}    +\frac{CE[|g_{1}^{\top}(X-\mu)|^{3}]}{\sigma^{3}\sqrt{n}},
\end{align*}
where $C$ is a universal constant and $C_{1}$ is a constant depending only on $q$.
\end{theorem}

The implication of Theorem \ref{CB_coverage_moments} on the choice of $p$ is similar to Theorem \ref{CB_coverage_subexp} so we omit it. In parallel to the above, explicit finite-sample bounds for standard quantile-based bootstrap methods can also be obtained by means of Theorem \ref{general_other_coverages} under the assumptions in Theorems \ref{CB_coverage_nonlinear}, \ref{CB_coverage_subexp} or \ref{CB_coverage_moments} (see Appendix A).

\section{Numerical Experiments\label{sec:num}}

We consider six high-dimensional examples:

\textbf{Ellipsoidal estimation:} The estimation target is $g(\mu)=||\mu||^2$, where $\mu$ is the mean of $X\in\mathbb R^p$ with ground-truth distribution $N(0.02\mathbf{1}_p,0.01I_{p\times p})$. Sample size $n=10^{5}$ and dimension $p=2.5\times10^{4}$.

\textbf{Sinusoidal estimation:} The estimation target is $g(\mu)=\sum_{i=1}^{p}\sin(\mu_{i})$, where $\mu=(\mu_i)_{i=1}^{p}$ is the mean of $X\in\mathbb R^p$ with ground-truth distribution $N(\mathbf{0},0.01I_{p\times p})$. Sample size $n=10^{5}$ and dimension $p=2.5\times10^{4}$.

\textbf{Linear regression:} Consider the true model $Y=X^{\top}\beta+\varepsilon$, where $X\in\mathbb{R}^{p}$ follows $N(\mathbf{0},0.01I_{p\times p})$ and $\varepsilon\sim N(0,1)$ independent of $X$. The first, second and last $1/3$ components of $\beta=(\beta_{i})_{i=1}^{p}$ are $0,2,-1$ respectively. We estimate $\beta$ given i.i.d. data $(X_{i},Y_{i})_{i=1}^{n}$ with $n=10^{5}$ and $p=9000$.

\textbf{Logistic regression:} Consider the true model $Y\in\{0,1\}$, $X\in\mathbb{R}^{p}$, $P(Y=1|X)=\exp(X^{\top}\beta)/(1+\exp(X^{\top}\beta))$, where $X\sim N(\mathbf{0},0.01I_{p\times p})$. The first $300$ components of $\beta_{i}$'s are $1$, the second $300$ components $-1$ and the rest $0$. As suggested in \cite{sur2019modern}, we choose such values of $\beta_{i}$'s to make sure $Var(X^{\top}\beta)=6$ does not increase with $p$ so that $P(Y=1|X)$ is not trivially equal to $0$ or $1$ in most cases. We estimate $\beta$ given i.i.d. data $(X_{i},Y_{i})_{i=1}^{n}$ with $n=10^{5}$ and $p=9000$.

\textbf{Stochastic simulation model:} Consider a stochastic computer communication model used to calculate the steady-state average message delay (\cite{cheng1997sensitivity,lin2015single,lam2021subsampling}; see Appendix C for details). This problem can be cast as computing $\psi(P_1,\ldots,P_p)$ where $\psi$ represents this expensive simulation model (due to the need to run very long time in order to reach steady state) and $P_j$'s denote the input distributions, $p=13$ in total. The data sizes for observing these 13 input models range from $3\times10^{4}$ to $6\times10^{4}$.

\textbf{A real data example:} We run logistic regression on the RCV1-v2 data in \cite{lewis2004rcv1}. This dataset contains $n=804414$ manually categorized newswire stories with a total of $p=47236$ features. ``Economics'' (``ECAT'') is chosen as the $+1$ label. We target coefficient estimation.

\textbf{Setups and comparison benchmarks. }
In each example above, our targets are $95\%$-level confidence intervals for the target parameters. We test four bootstrap confidence intervals: 1) cheap bootstrap \eqref{cheap CI}; 2) basic bootstrap described in Section \ref{sec:method}; 3) percentile bootstrap described in Section \ref{sec:method}; 4) standard error bootstrap that uses standard normal quantile and standard deviation of $\hat\psi_{n}^{\ast b}$'s in lieu of $t_{B,1-\alpha/2}$ and $S_{n,B}$ respectively in \eqref{cheap CI}. For each setup except the real-data example, we run $1000$ experimental repetitions, each time generating a new data set from the ground truth distribution and construct the intervals. We report the empirical coverage and average interval width over these repetitions. For examples with more than one target estimation quantity, we further average the coverages and widths over all these targets. We vary the number of resamples $B$ from 1 to 10 in all examples.

\textbf{Results and discussions. }Table \ref{num_table} describes our results, where we report $B=1,2,5,10$.

\emph{Coverage probability: }The cheap bootstrap performs the best in terms of the coverage probabilities in almost all cases (except the real-data example where we cannot validate and only report the interval widths). In all but two entries, the cheap bootstrap gives the closest coverages to the nominal 95\% among all considered bootstrap methods, and in all but three entries the cheap bootstrap coverages are above 95\%. In contrast, other approaches are substantially below the nominal level except for very few cases with $B=10$. For example, in the ellipsoidal estimation, cheap bootstrap coverage probabilities are above 95\% for all considered $B$'s, while the highest coverage among other bootstrap methods is 82.1\% even for $B=10$. These observations corroborate with theory since unlike standard bootstrap methods, the cheap bootstrap gives small coverage errors even with very small $B$. Note that when $B=1$, the entries of other bootstrap methods are all ``N.A.'' since quantile-based approaches cannot even output two distinct finite numbers using one resample, and standard error bootstrap uses $B-1$ in the denominator of the sample variance.

\emph{Interval width: }Cheap bootstrap intervals are wider than other bootstrap intervals. However, these widths appear to decay very fast for the first few $B$'s. In all examples, they decrease by around $2/3$ from $B=1$ to $B=2$ and by around $4/5$ from $B=1$ to $B=10$, hinting a quick enhancement of statistical efficiency as the computation budget increases. Note that while the other bootstrap intervals are shorter, they fall short in attaining the nominal coverage. It is thus reasonable to see the larger widths of the cheap bootstrap intervals which appear to push up the coverages by the right amounts.

\begin{table}
\centering
\caption{Coverage probabilities (Pro.) and confidence interval widths (Wid.) of the numerical examples. The closest coverage probability to the nominal 95\% level among all methods are \textbf{bolded}.}
\medskip
\label{num_table}
\resizebox{\columnwidth}{!}{
\begin{tabular}{cccccccccc}
\toprule
\multirow{2}{*}{Example} & \multirow{2}{*}{$B$} & \multicolumn{2}{c}{Cheap Bootstrap}                  & \multicolumn{2}{c}{Basic Bootstrap}         & \multicolumn{2}{c}{Percentile Bootstrap}         & \multicolumn{2}{c}{Standard Error Bootstrap}         \\
                         &                        & Pro.            & Wid.                  & Pro.   & Wid.                  & Pro.   & Wid.                  & Pro.   & Wid.                   \\
\midrule
~                        & 1                      & \textbf{96.0\%} & 0.069                 & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Ellipsoidal              & 2                      & \textbf{97.3\%} & 0.026                 & 32.2\% & 0.002                 & 5.5\%  & 0.002                 & 55.1\% & 0.006                  \\
estimation               & 5                      & \textbf{97.4\%} & 0.016                 & 66.0\% & 0.005                 & 13.6\% & 0.005                 & 70.1\% & 0.007                  \\
                         & 10                     & \textbf{97.5\%} & 0.014                 & 82.1\% & 0.006                 & 20.8\% & 0.006                 & 73.6\% & 0.008                  \\
\midrule
                         & 1                      & \textbf{94.4\%} & 0.958                 & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Sinusoidal               & 2                      & \textbf{95.2\%} & 0.384                 & 29.6\% & 0.051                 & 35.2\% & 0.051                 & 63.2\% & 0.142                  \\
estimation               & 5                      & \textbf{93.6\%} & 0.248                 & 71.2\% & 0.117                 & 66.4\% & 0.117                 & 86.4\% & 0.187                  \\
                         & 10                     & \textbf{94.4\%} & 0.222                 & 84.0\% & 0.156                 & 83.2\% & 0.156                 & 89.6\% & 0.196                  \\
\midrule
                         & 1                      & \textbf{95.1\%} & 0.68                  & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Linear                   & 2                      & \textbf{95.1\%} & 0.256                 & 33.5\% & 0.038                 & 33.5\% & 0.038                 & 70.2\% & 0.105                  \\
regression               & 5                      & \textbf{95.2\%} & 0.164                 & 67.0\% & 0.078                 & 67.0\% & 0.078                 & 88.1\% & 0.123                  \\
                         & 10                     & \textbf{95.2\%} & 0.146                 & 82.2\% & 0.103                 & 82.2\% & 0.103                 & 92.1\% & 0.128                  \\
\midrule
                         & 1                      & \textbf{96.1\%} & 2.866                 & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Logistic                 & 2                      & \textbf{96.9\%} & 1.074                 & 39.7\% & 0.147                 & 31.7\% & 0.147                 & 73.4\% & 0.407                  \\
regression               & 5                      & \textbf{97.9\%} & 0.685                 & 77.9\% & 0.302                 & 63.3\% & 0.302                 & 91.0\% & 0.479                  \\
                         & 10                     & 98.4\% & 0.609                 & 91.9\% & 0.400                 & 77.7\% & 0.400                 & \textbf{94.6\%} & 0.496                  \\
\midrule
                         & 1                      & \textbf{96.9\%} & 1.757$\times 10^{-3}$ & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Stochastic               & 2                      & \textbf{98.8\%} & 6.417$\times 10^{-4}$ & 21.9\% & 6.962$\times 10^{-5}$ & 47.0\% & 6.962$\times 10^{-5}$ & 68.7\% & 1.930$\times 10^{-4}$  \\
simulation               & 5                      & \textbf{99.7\%} & 4.044$\times 10^{-4}$ & 43.2\% & 1.428$\times 10^{-4}$ & 90.4\% & 1.428$\times 10^{-4}$ & 87.1\% & 2.269$\times 10^{-4}$  \\
                         & 10                     & 100\%  & 3.591$\times 10^{-4}$ & 55.6\% & 1.915$\times 10^{-4}$ & 99.8\% & 1.915$\times 10^{-4}$ & \textbf{92.6\%} & 2.375$\times 10^{-4}$  \\
\midrule
                         & 1                      & N.A.            & 3.594                 & N.A.   & N.A.                  & N.A.   & N.A.                  & N.A.   & N.A.                   \\
Real                     & 2                      & N.A.            & 1.361                 & N.A.   & 0.201                 & N.A.   & 0.201                 & N.A.   & 0.556                  \\
data                     & 5                      & N.A.            & 0.877                 & N.A.   & 0.414                 & N.A.   & 0.414                 & N.A.   & 0.658                  \\
                         & 10                     & N.A.            & 0.779                 & N.A.   & 0.547                 & N.A.   & 0.547                 & N.A.   & 0.682                  \\
\bottomrule
\end{tabular}
}
\end{table}

\section{Discussions and Other Connections}
We close this paper by discussing our results in the context of several lines of literature. First, our work is related to bootstrap coverage analysis. The commonest approach is to use the Edgeworth expansion that reveals the asymptotic higher-order terms in the coverage errors; see the comprehensive monograph \cite{hall2013bootstrap}. It is only until recently where finite-sample bounds appear, mostly in the high-dimensional CLT literature where the target is sample mean (\cite{chernozhukov2017central,lopes2020central,chernozhukov2020nearly} and references therein). They aim to prove a uniform finite-sample bound of normal approximation of the sample mean over all hyperrectangles. An alternative approach is to use Stein's method (\cite{fang2021high}).

Second, within the bootstrap framework, various approaches have been proposed to reduce the Monte Carlo sampling effort by, e.g., variance reduction such as importance sampling (\cite{booth1993simple}), or analytic approximation especially when applying iterated bootstraps (\cite{booth1994monte,lee1995asymptotic}). These methods, however, require additional knowledge such as an explicit way to calculate variance, or focus on tail estimation issue. The closest work to the cheap bootstrap idea we utilize in this paper is \cite{hall1986number} who investigates the number of resamples for one-sided basic bootstrap intervals. Nonetheless, \cite{hall1986number} suggests a minimum of 19 for $B$ in a 95\%-level interval, obtained via an order-statistics calculation.

Finally, there are approaches outside the bootstrap framework that are computationally light. The so-called batch means method (\cite{glynn1990simulation,schmeiser1982batch,schruben1983confidence}) divides data into batches to construct batched estimates which are then aggregated to generate confidence intervals. This approach has been used commonly in simulation analysis to tackle situations with serial dependence. However, it typically faces a tradeoff between the number of batches and the sample size per batch, which ideally should both be large to obtain good performances. Compared to batching, the cheap bootstrap that we consider in this paper is free of this tradeoff as the resample size can be chosen independent of the number of resamples.

\bibliographystyle{plainnat}

Methodology (stat.ME); Statistics Theory (math.ST); Computation (stat.CO)


Beta Laguerre ensembles in global regime

Abstract: Beta Laguerre ensembles which are generalizations of Wishart ensembles and Laguerre ensembles can be realized as eigenvalues of certain random tridiagonal matrices. Analogous to the Wishart ($\beta=1$) case and the Laguerre ($\beta = 2$) case, for fixed $\beta$, it is known that the empirical distribution of the eigenvalues of these ensembles converges weakly to Marchenko--Pastur distributions, almost surely. The paper restudies the limiting behavior of the empirical distribution but in regimes where the parameter $\beta$ is allowed to vary as a function of the matrix size $N$. We show that the above Marchenko--Pastur law holds as long as $\beta N \to \infty$. When $\beta N \to 2c \in (0, \infty)$, the limit is related to associated Laguerre orthogonal polynomials. Gaussian fluctuations around the limit are also studied.

\section{Introduction}
Beta Laguerre ($\beta$-Laguerre) ensembles are ensembles of $N$ positive particles distributed according  the following joint probability density function
\begin{equation}\label{bLE}
	\frac{1}{Z_{N, M}^{(\beta)}}\prod_{i < j}|\lambda_j - \lambda_i|^\beta \prod_{i = 1}^N \left( \lambda_i^{\frac{\beta}{2}(M-N+1) - 1} e^{-\frac{\beta M}{2}  \lambda_i} \right), \quad (\lambda_i > 0),
\end{equation}
where $\beta > 0$ and $M > N -1$ are parameters, and $Z_{N, M}^{(\beta)}$ is the normalizing constant. For two special values $\beta = 1,2$, they are the joint density of the eigenvalues of Wishart matrices and Laguerre matrices, respectively. For general $\beta > 0$, the ensembles can be realized as eigenvalues of a random tridiagonal matrix model $J_N = B_N (B_N)^t$ with a bidiagonal matrix $B_N$ consisting of independent entries distributed as
\[
	B_N = \frac{1}{\sqrt{\beta M}} \begin{pmatrix}
		\chi_{\beta M}	\\
		\chi_{\beta (N-1)}	&\chi_{\beta(M - 1)}	\\
		&\ddots	&\ddots\\
		&&\chi_\beta	&\chi_{\beta(M - N + 1)}
	\end{pmatrix},
\]
where $\chi_k$ denotes the chi distribution with $k$ degrees of freedom, and $(B_N)^t$ denotes the transpose of $B_N$.

Wishart matrices (resp.~Laguerre matrices) are  random matrices of the form $M^{-1}G^{t}G$ (resp.~$M^{-1}G^{*}G$), where $G$ is an $M \times N$ matrix consisting of i.i.d.~(independent identically distributed) entries of standard real (resp.~complex) Gaussian distribution.  Here $G^*$  denotes the  Hermitian conjugate of the matrix $G$. These random matrix models can be defined in a different way as invariant probability measures on the set of symmetric (resp.~Hermitian) matrices. The limiting behavior of their eigenvalues has been studied intensively, and it is known that as $N \to \infty$ with $N/M \to \gamma \in (0,1)$, the empirical distribution
\[
	L_N = \frac{1}{N}\sum_{i = 1}^N \delta_{\lambda_i}
\]
converges to the Marchenko--Pastur distribution with parameter $\gamma$, almost surely. Here $\delta_\lambda$ denotes the Dirac measure at $\lambda$. The convergence means that for any bounded continuous function $f$,
\[
	\int f(x) dL_N(x) = \frac{1}{N} \sum_{i = 1}^N f(\lambda_i) \to \int f(x) mp_\gamma(x) dx\quad \text{as}\quad N \to \infty, \text{almost surely,}
\]
where $mp_\gamma(x)$ is the density of the Marchenko--Pastur distribution with parameter $\gamma$,
\[
	mp_\gamma(x) = \frac{1}{2\pi \gamma x} \sqrt{(\lambda_+ - x)(x - \lambda_-)}, \quad (\lambda_- < x < \lambda_+), \quad \lambda_{\pm} = (1 \pm \sqrt \gamma)^2.
\]
Gaussian fluctuations around the limit were also established with explicit formula for the limiting variance. The convergence to a limit and fluctuations around the limit of the empirical distributions are two typical problems in the global regime when a random matrix model is considered. Refer the book \cite{Pastur-book} for more details on Wishart and Laguerre matrices.

The convergence to Marchenko--Pastur distributions and fluctuations around the limit were extended to $\beta$-Laguerre ensembles for general $\beta> 0$ in \cite{DE06} by using the random tridiagonal matrix model. Based on the model as well, results in the local regime and the edge regime were established \cite{Jacquot-Valko-2011, Sosoe-Wong-2014}. Note that the parameter $\beta$ is fixed in those studies.

The aim of this paper is to refine results in the global regime of $\beta$-Laguerre ensembles. We assume that the parameter $\beta$ varies as a function of $N$, while the parameter $M$ depends on $N$ in a way that $N/M \to \gamma \in (0,1)$ as usual. We show that the Marchenko--Pastur law holds as long as $\beta N \to \infty$. When $\beta N $ stays bounded, the limiting measure is related to associated Laguerre orthogonal polynomials. For the proof, we extend some ideas that were used in \cite{DS15, Nakano-Trinh-2018, Trinh-2017} in case of Gaussian beta ensembles. Our main results are stated in the following two theorems.

\begin{theorem}[Convergence to a limit distribution]\label{thm:LLN}
\begin{itemize}
	\item[\rm(i)] As $\beta N \to \infty$, the empirical distribution $L_N$ converges weakly to the Marchenko--Pastur distribution with parameter $\gamma$, almost surely. Here $L_N = N^{-1} \sum_{i = 1}^N \delta_{\lambda_i}$ is the empirical distribution of the $\beta$-Laguerre ensembles~\eqref{bLE}.

	\item[\rm(ii)] As $\beta N \to 2c \in (0,\infty)$, the sequence $\{L_N\}$ converges weakly to the probability measure $\nu_{\gamma, c}$, almost surely, where $\nu_{\gamma, c}$ is given in Theorem~{\rm\ref{thm:2c-unscale}}.
\end{itemize}
\end{theorem}

To prove the above results, it is sufficient to show that any moment of $L_N$ converges almost surely to the corresponding moment of $\nu_{\gamma, c}$. Here for convenience, we refer to $\nu_{\gamma, \infty}$ as the Marchenko--Pastur distribution with parameter $\gamma$. It then follows that (see \cite{Trinh-ojm-2018}) for any continuous function $f$ of polynomial growth (there is a polynomial $P(x)$ such that $|f(x)| \le P(x)$ for all $x \in \R$),
\[
	\int f(x) dL_N(x) = \frac{1}{N} \sum_{i = 1}^N f(\lambda_i) \to \int f(x) d\nu_{\gamma, c}( x)\quad \text{as}\quad N \to \infty, \text{almost surely.}
\]

\begin{theorem}[Gaussian fluctuations around the limit] \label{thm:CLT-full}
Assume that the function $f$ has continuous derivative of polynomial growth. Then the following hold.
	\begin{itemize}
		\item[\rm(i)]	As $\beta N \to \infty$,
		\[
			\sqrt{\beta} \left( \sum_{i = 1}^N f(\lambda_i) - \Ex\bigg[ \sum_{i = 1}^N f(\lambda_i) \bigg]  \right) \dto \Normal(0, \sigma_f^2),
		\]
		where
\[
	\sigma_f^2 = \frac{1}{2\pi^2} \int_{\gamma_-}^{\gamma^+}\int_{\gamma_-}^{\gamma^+} \left(\frac{f(y) - f(x)}{y - x}\right)^2 \frac{4\gamma - (x - \gamma_m)(y - \gamma_m)}{\sqrt{4 \gamma - (x - \gamma_m)^2} \sqrt{4 \gamma - (y - \gamma_m)^2}}dx dy,
\]
with $\gamma_m = (\gamma_{-} + \gamma_{+})/2 = (1 + \gamma)$. Here `$\dto$' denotes the convergence in distribution.
		\item[\rm(ii)] As $\beta N \to 2c \in (0, \infty)$,
		\[
			\sqrt{\beta} \left( \sum_{i = 1}^N f(\lambda_i) - \Ex\bigg[ \sum_{i = 1}^N f(\lambda_i) \bigg]  \right) \dto \Normal(0, \sigma_{f,c}^2),
		\]
		where $\sigma_{f,c}^2$ is a constant.
	\end{itemize}
\end{theorem}

The paper is organized as follows. In the next section, we introduce the random tridiagonal matrix model and related concepts. Results on convergence to a limit and Gaussian fluctuations around the limit are shown in Section~3 and Section~4, respectively.

\section{Random tridigonal  matrix model and spectral measures}
\subsection{Random tridiagonal matrix model}

Let $G$ be an $M\times N$ random matrix consisting of i.i.d.\ real standard Gaussian random variables. Then $X = M^{-1} G^t G$ is called a Wishart matrix. When  $M \ge N$, the eigenvalues $\lambda_1, \dots, \lambda_N$ of $X$ have the following joint density
\[
	\frac{1}{Z_{M, N}} |\Delta(\lambda)| \prod_{i = 1}^N \left( \lambda_i^{\frac12(M - N + 1) - 1} e^{-\frac M2 \lambda_i} \right),\quad (\lambda_i > 0),
\]
where $Z_{M,N}$ is the normalizing constant, and $\Delta(\lambda)=\prod_{i < j}(\lambda_j - \lambda_i)$ denotes the Vandermonde determinant.
A Laguerre matrix $X = M^{-1} G^* G$ is obtained when the entries of $G$ are i.i.d.\ of  standard complex Gaussian distribution, where recall that $G^*$ stands for the Hermitian conjugate of $G$. In this case, the joint density of the eigenvalues has an analogous formula to the Wishart case. Then $\beta$-Laguerre ensembles are defined to be ensembles of $N$ positive particles with the joint density
\begin{equation}\label{bLE-2}
	\frac{1}{Z_{M, N}^{(\beta)}} |\Delta(\lambda)|^\beta \prod_{i = 1}^N \left( \lambda_i^{\frac\beta2(M - N + 1) - 1} e^{-\frac {\beta M}{2} \lambda_i} \right),\quad (\lambda_i > 0),
\end{equation}
where $\beta > 0$ and $M > N - 1$, which generalizes Wishart $(\beta = 1)$ and Laguerre $(\beta = 2)$ ensembles.

A random tridiagonal matrix model for $\beta$-Laguerre ensembles was introduced in \cite{DE02} based on tridiagonalizing Wishart or Laguerre matrices. Let
\[
	B_N = \frac{1}{\sqrt{\beta M}} \begin{pmatrix}
		\chi_{\beta M}	\\
		\chi_{\beta (N-1)}	&\chi_{\beta(M - 1)}	\\
		&\ddots	&\ddots\\
		&&\chi_\beta	&\chi_{\beta(M - N + 1)}
	\end{pmatrix}
\]
denote a random bidiagonal matrix with independent entries. Then the eigenvalues of the tridiagonal matrix $J_N = B_N (B_N)^t$ are distributed as the $\beta$-Laguerre ensembles~\eqref{bLE-2}. Using this random matrix model and combinatorial arguments, the convergence to Marchenko--Pastur distributions and Gaussian fluctuations around the limit (only for polynomial test functions) were established in \cite{DE06}.

\subsection{Spectral measures of Jacobi matrices}

A symmetric tridiagonal matrix is called a Jacobi matrix.
Spectral measures of Jacobi matrices $\{J_N\}$ have been studied recently. Here the spectral measure of $J_N$ is defined to be the probability measure $\mu_N$ satisfying
\[
	\int x^k d\mu_N(x) = (J_N)^k(1,1), \quad k = 0,1,2,\dots.
\]
It follows from the spectral decomposition of $J_N$ that $\mu_N$ has the following form
\[
	\mu_N = \sum_{i = 1}^N q_i^2 \delta_{\lambda_i},\quad (q_i^2 = v_i(1)^2),
\]
where $v_1, \dots, v_N$ are the normalized eigenvectors of $J_N$ corresponding to the eigenvalues $\lambda_1, \dots, \lambda_N$.

Spectral measures can also be defined for infinite Jacobi matrices. Let $J$ be an infinite Jacobi matrix,
\[
	J = \begin{pmatrix}
		a_1	&b_1	\\
		b_1	&a_2		&b_2\\
		&\ddots	&\ddots	&\ddots
	\end{pmatrix}, \quad a_i \in \R, b_i > 0.
\]
Then there exists a probability measure $\mu$ such that
\[
	\int x^k d\mu(x) = J^k(1,1), \quad k = 0,1,\dots.
\]
When the above moment problem has a unique solution $\mu$, the measure $\mu$ is called the spectral measure of $J$. A simple, but useful sufficient condition for the uniqueness is given by \cite[Corollary 3.8.9]{Simon-book-2011}
\[
	\sum_{n=1}^\infty \frac{1}{b_n} = \infty.
\]

By definition, moments of the spectral measure $\mu_N$ depend locally on upper left entries of $J_N$, and thus, the limiting behavior of $\mu_N$ follows easily from those of entries. In particular, for fixed $\beta$, as $N \to \infty$ with $N/M \to \gamma \in (0,1)$, entry-wisely,
\[
	B_N = \frac{1}{\sqrt{\beta M}} \begin{pmatrix}
		\chi_{\beta M}	\\
		\chi_{\beta (N-1)}	&\chi_{\beta(M - 1)}	\\
		&\ddots	&\ddots\\
		&&\chi_\beta	&\chi_{\beta(M - N + 1)}
	\end{pmatrix}
	\to \begin{pmatrix}
		1	\\
		\sqrt{\gamma}	&1	\\
		&\ddots	&\ddots\\

	\end{pmatrix}.
\]
Here the convergence holds almost surely and in $L_q$ for any $q \in [1, \infty)$. It follows that almost surely, the spectral measure $\mu_N$ converges weakly to the spectral measure of the following infinite Jacobi matrix
\begin{equation}\label{MP-gamma}
	MP_\gamma = \begin{pmatrix}
		1\\
		\sqrt{\gamma}		&1\\
		&\ddots 	&\ddots
	\end{pmatrix}
	\begin{pmatrix}
		1	&\sqrt{\gamma}\\
		&1	&\sqrt{\gamma}		\\
		&&\ddots 	&\ddots
	\end{pmatrix}
	=
	\begin{pmatrix}
		1		&\sqrt{\gamma}	\\
		\sqrt{\gamma}		&1+\gamma		&\sqrt{\gamma}\\
		&\sqrt{\gamma}		&1+\gamma		&\sqrt{\gamma}\\
		&&\ddots 	&\ddots 	&\ddots
	\end{pmatrix},
\end{equation}
which is nothing but the Marchenko--Pastur distribution with parameter $\gamma$ \cite{Trinh-ojm-2018}.

For the Jacobi matrix $J_N$, the weights $q_1^2, \dots, q_N^2$ in the spectral measure $\mu_N$ are independent of the eigenvalues and have Dirichlet distribution with parameter $\beta/2$. From which, the empirical distribution $L_N$ and the spectral measure $\mu_N$ have the following relations
\begin{align}
	 \Ex[\bra{L_N, f}] &= \Ex[\bra{\mu_N, f}] , \label{same-mean}\\
	\Var[\bra{L_N, f}] &= \frac{\beta N + 2}{\beta N} \Var[\bra{\mu_N, f}] - \frac{2}{\beta N} \left(\Ex[\bra{\mu_N, f^2}] - \Ex[\bra{\mu_N, f}]^2 \right), \label{relation-of-variance}
\end{align}
for suitable test functions $f$. Here we use the notation $\bra{\mu, f}$ to denote the integral $\int f d\mu$ for a measure $\mu$ and an integrable function $f$.

We conclude this section by giving several remarks on spectral measures of Jacobi matrices. The spectral measure $\mu$ orthogonalizes the sequence of polynomials $\{P_n\}_{n \ge 0}$ defined by
\begin{align*}
	&P_0 = 1, \quad P_1 = x - a_1,\\
	&P_{n+1} = x P_n - a_{n + 1}P_n - b_n^2 P_{n - 1}, \quad n \ge 1.
\end{align*}
In a particular case, $a_n =(\alpha + 2n-1)$ and $b_n = \sqrt{n}\sqrt{\alpha + n}$, the sequence of polynomials  $\{L_n\}$ defined by
\begin{align*}
	&L_0 = 1, \quad L_1 = x - (\alpha + 1),\\
	&L_{n+1} = x L_n - (\alpha + 2n + 1)L_n - n(\alpha + n) L_{n - 1}, \quad n \ge 1,
\end{align*}
coincides with the sequence of scaled generalized Laguerre polynomials. The spectral measure in this case is the Gamma distribution with parameters $(\alpha+1, 1)$, that is, the probability measure with density $\Gamma(\alpha + 1)^{-1} x^\alpha e^{-x}, x > 0$. In other words, the Gamma distribution with parameters $(\alpha + 1,1)$ is the spectral measure of the following infinite Jacobi matrix $J_\alpha$,
\begin{align*}
	J_\alpha &= \begin{pmatrix}
		\alpha + 1	& \sqrt{\alpha + 1} \\
		\sqrt{\alpha + 1} 	&\alpha + 3	& \sqrt{2}\sqrt{ \alpha + 2} 	\\
		&&\ddots	&\ddots	&\ddots
	\end{pmatrix} \\
	&=	\begin{pmatrix}
		\sqrt{\alpha + 1}	\\
		\sqrt 1	&\sqrt{\alpha + 2}	\\

		&\ddots	&\ddots
	\end{pmatrix}
	\begin{pmatrix}
		\sqrt{\alpha + 1}	&\sqrt1	\\
			&\sqrt{\alpha + 2}	&\sqrt{2}	\\

			&&\ddots	&\ddots
	\end{pmatrix}.
\end{align*}

When the entries of $J_\alpha$ are `shifted' by a constant $c \in \R$, the resulting orthogonal polynomials are called associated Laguerre polynomials. The spectral measure in this case was explicitly calculated in \cite{Ismail-et-al-1988} as Model II for associated Laguerre orthogonal polynomials.

\begin{lemma}[\cite{Ismail-et-al-1988}]\label{lem:associated-Laguerre}
For $c > -1$ and $\alpha + c + 1 > 0$, let
\[
	J_{\alpha, c} =
	\begin{pmatrix}
		\sqrt{\alpha + c + 1} \\
		\sqrt{c + 1}	& \sqrt{\alpha + c + 2}\\
		&\ddots	&\ddots
	\end{pmatrix}
	\begin{pmatrix}
		\sqrt{\alpha + c + 1} & \sqrt{c + 1}	\\
		&\sqrt{\alpha + c + 2}	&\sqrt{c+2}\\
		&&\ddots	&\ddots
	\end{pmatrix},
\]
and $\mu_{\alpha,c}$ be its spectral measure. Then the density and the Stieltjes transform of $\mu_{\alpha, c}$ are given by
\begin{align*}
	\mu_{\alpha, c}(x) &= \frac{1}{\Gamma(c+1) \Gamma(1+c+\alpha)} \frac{x^{\alpha} e^{-x}}{|\Psi(c, -\alpha; x e^{-i \pi})|^2}, \quad x \ge 0,\\
	S_{\mu_{\alpha, c}}(z) &= \int_0^\infty \frac{\mu_{\alpha, c} (x) dx}{x - z} = \frac{\Psi(c+1, 1 -\alpha; -z)}{\Psi(c, -\alpha; -z)}, \quad z \in \C \setminus \R.
\end{align*}
Here $\Psi(a, b; z)$ is Tricomi's confluent hypergeometric function.
\end{lemma}

Note that when $\alpha$ is not an integer, an alternative formula for $\Psi(a, b; z)$ could be used
\begin{align*}
	\Psi(c,  - \alpha; xe^{-i \pi}) &=  \frac{\Gamma(\alpha + 1)}{\Gamma(\alpha + c + 1)} \,_1F_1(c; -\alpha; -x) \\
	&\qquad - \frac{\Gamma(-\alpha - 1)}{\Gamma(c)} x^{\alpha + 1} e^{-i\pi \alpha} \,_1F_1 (\alpha + c + 1; 2+\alpha; -x) ,
\end{align*}
where ${}_1F_1(a; b; z)$ is the Kummer function.

\section{Convergence to a limit distribution}
In what follows, the parameter $M$ depends on $N$ in the way that $N/M \to \gamma \in (0,1)$ as $N \to \infty$. We study the limiting behavior of the empirical distribution $L_N$ through its moments $\bra{L_N, x^r}, r=0,1,2,\dots$. Recall from the identity \eqref{same-mean} that for $r = 0,1,2,\dots,$
\[
	\Ex[\bra{L_N, x^r}] = \Ex \bigg[\frac1N \tr [(J_N)^r] \bigg] = \Ex[\bra{\mu_N, x^r}] = \Ex[(J_N)^r(1,1)].
\]
Here $\tr[A]$ denotes the trace of a matrix $A$.

\subsection{The Marchenko--Pastur regime, $\beta N \to \infty$}
A key observation in this regime is the following asymptotic behavior of chi distributions.
\begin{lemma}
As $k \to \infty$,
	\[
		\frac{\chi_k}{\sqrt{k}} \to  1 \quad \text{ in $L^q$ for any $1 \le q < \infty$.}
	\]

\end{lemma}

Let $\{c_i\}_{i = 1}^N$ and $\{d_i\}_{i = 1}^{N-1}$ be the diagonal and the sub-diagonal of $B_N$, respectively. Note that although we do not write explicitly, $\{c_i\}$ and $\{d_i\}$ depend on the triple $(N, M, \beta)$.
It is clear that as $\beta N \to \infty$,
\[
	B_N = \frac{1}{\sqrt{\beta M}} \begin{pmatrix}
		\chi_{\beta M}	\\
		\chi_{\beta (N-1)}	&\chi_{\beta(M - 1)}	\\
		&\ddots	&\ddots\\
		&&\chi_\beta	&\chi_{\beta(M - N + 1)}
	\end{pmatrix}
	\to \begin{pmatrix}
		1	\\
		\sqrt{\gamma}	&1	\\
		&\ddots	&\ddots\\

	\end{pmatrix}.
\]
Here the convergence means the pointwise convergence of entries, which holds in $L^q$ for any $q \in [1, \infty)$, that is, for fixed $i$, as $N \to \infty$ with $\beta N \to \infty$,
\begin{equation}\label{Lp-convergence-of-cd}
	c_i \to 1, \quad d_i \to \sqrt{\gamma}	\quad \text{in $L^q$ for $q\in [1, \infty)$.}
\end{equation}

Since $J_N$ is a tridiagonal matrix of the form
\[
	J_N = B_N (B_N)^t = \begin{pmatrix}
		c_1^2	&c_1d_1\\
		c_1d_1	&c_2	^2 + d_1^2 	&c_2d_2\\
		&\ddots	&\ddots	&\ddots\\
		&&c_{N-1}d_{N - 1}	&c_N^2 + d_{N-1}^2
	\end{pmatrix},
\]
it follows that for fixed $r \in \N$, when $N>r$, $(J_N)^r(1,1)$ is a polynomial of $\{c_i, d_i\}_{i \le \frac{r+1}2}$. Let us show some explicit formulae for $(J_N)^r(1,1)$,
\begin{align*}
	J_N(1,1) &= c_1^2,\\
	(J_N)^2(1,1) &= c_1^4 + c_1^2 d_1^2,\\
	(J_N)^3(1,1) &= c_1^6 + 2 c_1^4 d_1^2 + c_1^2 c_2^2 d_1^2 + c_1^2 d_1^4.
\end{align*}
Then the pointwise convergence \eqref{Lp-convergence-of-cd} implies that as $N \to \infty$ with $\beta N \to \infty$,
\[
	(J_N)^r(1,1) \to (MP_\gamma)^r(1,1)		\quad \text{in $L^q$ for any $q \in [1,\infty)$,}
\]
where recall that the Jacobi matrix $MP_\gamma$ is given in \eqref{MP-gamma}.
Consequently, as $N \to \infty$,
\[
	\Ex[(J_N)^r(1,1) ] \to  (MP_\gamma)^r(1,1).
\]

Recall also that the spectral measure of $MP_\gamma$ is the Marchenko--Pastur distribution with parameter $\gamma$. Therefore,  in this regime, the following convergence of the mean values holds.
\begin{lemma}\label{lem:mean-infinity}
	For any $r \in \N$, as $N \to \infty$ with $\beta N \to \infty$,
	\[
		\Ex[\bra{L_N, x^r}] = \Ex\left[\frac1N  \tr[ (J_N)^r] \right ]  = \Ex[(J_N)^r(1,1) ] \to \bra{mp_\gamma, x^r}.
	\]
\end{lemma}

\subsection{High temperature regime, $\beta N \to 2c \in (0, \infty)$}

Let
\[
	B_\infty :=\begin{pmatrix}
		\chitilde_{\frac{2c}\gamma}		\\
		\chitilde_{2c}	&\chitilde_{\frac{2c}\gamma}\\
		&\ddots 	&\ddots
	\end{pmatrix}, \quad
		\chitilde_k = \frac{1}{\sqrt2} \chi_k,
\]
be an infinite bidiagonal matrix with independent entries.
Then as $\beta N \to 2c$,
\[
	B_N = \frac{1}{\sqrt{\beta M}} \begin{pmatrix}
		\chi_{\beta M}	\\
		\chi_{\beta (N-1)}	&\chi_{\beta(M - 1)}	\\
		&\ddots	&\ddots\\
		&&\chi_\beta	&\chi_{\beta(M - N + 1)}
	\end{pmatrix}
	\dto \frac{\sqrt{\gamma}}{\sqrt c}B_\infty,
\]
meaning that entries of $B_N$ converge in distribution to the corresponding entries of the infinite matrix $\frac{\sqrt{\gamma}}{\sqrt c}B_\infty$. Recall that $(J_N)^r(1,1)$ is a polynomial of $\{c_i, d_i\}_{i \le \frac{r+1}2}$. Moreover, the entries of $B_N$ and those of $B_\infty$ are independent. Therefore we can deduce that as $\beta N \to 2c$,
\begin{align*}
	(J_N)^r(1,1) &\dto (J_\infty)^r(1,1),\\
	\Ex[(J_N)^r(1,1) ] &\to \Ex[(J_\infty)^r(1,1) ],
\end{align*}
where
\[
	J_\infty = \frac{\gamma}{c} B_\infty (B_\infty)^t.
\]
By this approach, we have just shown the existence of the limit of the mean values when $\beta N \to 2c$. However, we are not able to identify the limit directly from $J_\infty$.

To identify the limit, we now extend some ideas that used in \cite{DS15}  in case of Gaussian beta ensembles to show the following.
\begin{theorem}\label{thm:2c-unscale}
	Let $\nu_{\gamma, c}$ be the spectral measure of the following Jacobi matrix
\[
	\frac{\gamma}{c}
	 \begin{pmatrix}
		\sqrt{\frac{c}\gamma } \\
		\sqrt{c + 1}	& \sqrt{\frac c{\gamma} + 1}\\
			&\ddots	&\ddots
	\end{pmatrix}
	 \begin{pmatrix}
		\sqrt{\frac{c}\gamma }	&\sqrt{c + 1} \\
		& \sqrt{\frac c{\gamma} + 1}	&\sqrt{c + 2}\\
		&&\ddots	&\ddots
	\end{pmatrix} = \frac \gamma c J_{\frac c\gamma, c}.
\]
Then for any $r \in \N$, as $N \to \infty$ with $\beta N \to 2c$,
\[
	\Ex[\bra{L_N, x^r}] = \Ex \left[ \frac1N \tr [(J_N)^r ] \right] = \Ex[(J_N)^r(1,1) ] \to \bra{\nu_{\gamma, c} , x^r}.
\]
\end{theorem}

Theorem~\ref{thm:2c-unscale} is equivalent to Theorem~\ref{thm:2c-scale} below which states a result for scaled $\beta$-Laguerre ensembles. Let us switch to the scaled version. Let $\tilde J_N := \tilde B_N (\tilde B_N)^t$ be a Jacobi matrix, where
\[
	\tilde B_N = \begin{pmatrix}
		\chitilde_{2\alpha + 2 + 2\kappa(N-1)}	\\
		\chitilde_{2\kappa (N-1)}	&\chitilde_{2\alpha + 2 + 2\kappa(N-2)}		\\
		&\ddots	&\ddots	\\
		&&\chitilde_{2\kappa}	&\chitilde_{2\alpha + 2}
	\end{pmatrix},
\]
(with $\kappa = \beta / 2 $ and $ \alpha = \frac{\beta}{2}(M - N + 1) - 1 = \kappa(M - N + 1) - 1)$.
Then the joint density of the eigenvalues of $\tilde J_N$ is proportional to
\[
	 |\Delta(\tilde\lambda)|^{2\kappa} \prod_{i = 1}^N \left( \tilde\lambda_i^{\alpha} e^{-\tilde\lambda_i} \right),\quad \tilde \lambda_i > 0.
\]

Let $\tilde \mu_N$ be the spectral measure of $\tilde J_N$ and
\[
m_r(N, \kappa, \alpha) =  \Ex[\bra{\tilde\mu_N, x^r}] = \Ex[(\tilde J_N)^r(1,1)].
\]
Then $m_r(N, \kappa, \alpha)$ satisfies the following duality relation.
\begin{lemma}[cf.~{\cite[Theorem~2.11]{DE06}}]
The function $m_r(N, \kappa, \alpha)$ is a polynomial with respect to $N, \kappa$ and $\alpha$ and satisfies the following relation
	\[
		m_r(N, \kappa, \alpha) = (-1)^r \kappa^r m_r(- \kappa N, \kappa^{-1}, -\alpha/\kappa).
	\]
\end{lemma}

Based on the above duality relation, we now identify the limit of $m_r(N, \kappa, \alpha) $ in the regime where $\kappa N \to c$. For fixed $N$, it is straightforward to calculate the limit of $\kappa^{-1/2} \tilde B_N$ with parameters $(N, \kappa, a \kappa)$ as $\kappa \to \infty$, where $a$ is fixed,
\[
	\left( \frac{1}{\sqrt{\kappa}} \tilde B_N \right) \to \begin{pmatrix}
		\sqrt{a + N - 1} \\
		\sqrt{N - 1}	& \sqrt{a + N - 2}\\
		&\ddots	&\ddots\\
		&&\sqrt{1}	&\sqrt{a}
	\end{pmatrix} =: D_N(a).
\]
Here the convergence holds in $L^q$ entry-wisely. Therefore,
\[
	\lim_{\kappa \to \infty} \kappa^{-r} m_r(N, \kappa, a \kappa)  = \lim_{\kappa \to \infty} \Ex[\kappa^{-r}  (\tilde J_N)^r(1,1)] = (D_N(a) D_N(a)^t)^r(1,1).
\]

Next, in viewing of the duality relation, it holds that
\[
\lim_{N \to \infty, \kappa N \to c} m_r(N, \kappa, \alpha)=	\lim_{\kappa \to \infty} (-1)^r \kappa^{-r} m_r(- c, \kappa, -\alpha \kappa).
\]
Let us consider the  following infinite matrix by exchanging $N \leftrightarrow -c, a \leftrightarrow -\alpha$ and replacing the sign inside square roots as well,
\[
	W_{\alpha, c}=
	 \begin{pmatrix}
		\sqrt{\alpha + c + 1} \\
		\sqrt{c + 1}	& \sqrt{\alpha + c + 2}\\
		&\sqrt{c + 2}	&\sqrt{\alpha + c + 3} \\
		&&\ddots	&\ddots
	\end{pmatrix}.
\]
Let $J_{\alpha, c} = W_{\alpha, c} W_{\alpha, c}^t$ and $l_r(\alpha, c)= (J_{\alpha, c})^r(1,1)$.
Then it follows that
\[
	\lim_{\kappa N \to c} m_r(N, \kappa, \alpha) = \lim_{\kappa \to \infty} (-1)^r \kappa^{-r} m_r(- c, \kappa, -\alpha \kappa) = l_{r}(\alpha,c).
\]
In conclusion, we have just proved the following result.

\begin{theorem}\label{thm:2c-scale}
Let $\alpha > - 1$ and $\cc \ge 0$ be given. Then in the regime where $ \beta N \to 2 \cc$,
\[
	\Ex\left[ \frac 1N \tr ((\tilde J_N)^r)\right] = \Ex[(\tilde J_N)^r(1,1)] \to \bra{\mu_{\alpha, c}, x^r},
\]
for any $r \in \N$. Here recall that $\mu_{\alpha, c}$ is the spectral measure of $J_{\alpha, c}$ whose density is given in Lemma~{\rm\ref{lem:associated-Laguerre}}
\end{theorem}

The limiting measure in this regime was calculated in \cite{Allez-Wishart-2013} by a different method.

\subsection{Almost sure convergence}
In this section, we complete the proof of Theorem~\ref{thm:LLN} by showing the following almost sure convergence.
\begin{lemma}
For any $r \in \N$, as $N\beta \to 2c \in (0, \infty]$,
\[
	S_N := \frac{1}{N} \tr[(J_N)^r] - \Ex\left [\frac{1}{N} \tr[(J_N)^r] \right] \to 0,\quad \text{almost surely.}
\]
\end{lemma}
The idea is that for fixed $r$, $p_i := (J_N)^r(i, i)$ is independent of $p_j=(J_N)^r(j, j)$, if $|i - j| \ge D_r$, where $D_r$ is a constant. Then write $S_N$ as a sum of $D_r$ sums of independent random variables
\[
	S_N = \frac{1}{N}\sum_{i} (p_{1+i D_r} - \Ex[p_{1+i D_r}])   + \cdots + \frac{1}{N}\sum_{i} (p_{D_r+i D_r} - \Ex[p_{D_r+i D_r} ]).
\]
For each sum of independent random variables, we use the following result whose proof can be found in the proof of Theorem 2.3 in \cite{Trinh-2017} to show the almost sure convergence.
\begin{lemma}
Assume that for each $N$, the random variables $\{\xi_{N, i}\}_{i = 1}^{\ell_N}$ are independent and that
\begin{equation}\label{4-moment}
	\sup_{N} \sup_{1\le i \le \ell_N} \Ex[(\xi_{N, i})^4] < \infty.
\end{equation}
Assume further that $\ell_N / N \to const \in (0, \infty)$ as $N \to \infty$. Then as $N \to \infty$,
\[
	\frac{1}N \sum_{i = 1}^{\ell_N} \left(\xi_{N, i} - \Ex[\xi_{N, i}] \right) \to 0, \quad \text{almost surely}.
\]
\end{lemma}

The moment condition \eqref{4-moment} can be easily checked for $p_i$ in the regime $\beta N \to 2c \in (0, \infty]$. The desired almost sure convergence then follows immediately.

\section{Gaussian fluctuations around the limit}

\subsection{Polynomial test functions}

In this section,
we establish central limit theorems (CLTs) for polynomial test functions. Since arguments are similar to those used in \cite{Trinh-2017} in case of Gaussian beta ensembles, we only sketch main steps. Without loss of generality, assume that $M = N/\gamma$, where $\gamma \in (0,1)$ is fixed.

Let us write $B_N$ as
\[
	B_N = \frac{\sqrt{\gamma}}{\sqrt{\beta N}}\begin{pmatrix}
		c_1	\\
		d_1	&c_2	\\
		&\ddots	&\ddots\\
		&&d_{N - 1}	&c_N
	\end{pmatrix} ,\quad \text{where }
	\begin{cases}
		c_i \sim \chi_{\frac{\beta N}{\gamma} - \beta(i - 1)} ,\\
		d_i \sim \chi_{\beta N -  \beta i}.
	\end{cases}
\]
Then
\[
	J_N = B_N (B_N)^t = \frac{\gamma}{\beta N}\begin{pmatrix}
		c_1^2	&c_1d_1\\
		c_1d_1	&c_2	^2 + d_1^2 	&c_2d_2\\
		&\ddots	&\ddots	&\ddots\\
		&&c_{N-1}d_{N - 1}	&c_N^2 + d_{N-1}^2
	\end{pmatrix} .
\]
Recall that for fixed $r \ge 1$,  the $r$th moment $\bra{\mu_N, x^r}$ is a polynomial of $\{c_i, d_i\}_{1 \le i \le \frac{r+1}{2}}$. It is actually a polynomial of $\{c_i^2, d_i^2\}$, that is,
\[
	\bra{\mu_N, x^r} = (J_N)^r(1,1) =  \frac{\gamma^r}{(\beta N)^r} \sum_{\vec\eta, \vec\zeta} a(\vec\eta, \vec\zeta)  \prod_{i=1}^r c_i^{2\eta_i} d_i^{2\zeta_i},
\]
where non-negative integer vectors $\vec\eta = (\eta_1, \dots, \eta_r)$ and $\vec\zeta = (\zeta_1, \dots, \zeta_r)$ satisfy $\sum_{i = 1}^r (\eta_i + \zeta_i) = r$. Then, from formulae for moments of chi-squared distributions, we conclude that
\begin{lemma}
\begin{itemize}
\item[\rm(i)] For $r \in \N$,
\[
	\Ex[\bra{\mu_N, x^r}] = \sum_{k = 0}^r \frac{R_{r; k} (\beta)}{(\beta N)^k},
\]
where $R_{r; k} (\beta)$ is a polynomial in $\beta$ of degree at most $k$.

\item[\rm(ii)] For $r, s \in \N$,
\[
	\Ex[\bra{\mu_N, x^r}\bra{\mu_N, x^s}] = \sum_{k = 0}^{r+s} \frac{Q_{r,s; k} (\beta)}{(\beta N)^k},
\]
where $Q_{r,s; k} (\beta)$ is a polynomial in $\beta$ of degree at most $k$.
\end{itemize}
\end{lemma}

Let $p$ be a polynomial of degree $m$. From the above expressions, we can derive a general form of $\Var[\bra{\mu_N, p}]$, and then that of $\Var[\bra{L_N, p}]$ by taking into account of the relation \eqref{relation-of-variance}. Similar to the case of Gaussian beta ensembles, the formula for $\Var[\bra{L_N, p}]$ can be simplified as follows. The proof is omitted.
\begin{lemma}
Let $p$ be a polynomial of order $m$. Then the variance $\Var[\bra{L_N, p}] $ can be expressed as
\[
	\Var[\bra{L_N, p}]  = \sum_{k = 2}^{2m + 1} \frac{\beta \ell_{p; k} (\beta)}{(\beta N)^k},
\]
where $ \ell_{p; k} (\beta)$ is a polynomial in $\beta$ of degree at most $(k -2)$.
\end{lemma}

\begin{corollary}
\begin{itemize}
\item[\rm(i)]
	As $\beta N \to \infty$,
	\[
		\beta N^2 \Var[\bra{L_N, p}]  \to \sigma_p^2.
	\]
\item[\rm(ii)] As $\beta N \to 2c$,
	\[
		\beta N^2 \Var[\bra{L_N, p}]  \to \sigma_{p,c}^2.
	\]
\end{itemize}
\end{corollary}

Theorem 3.4 in \cite{Trinh-2017} provides sufficient conditions under which CLTs for $\{\bra{L_N, p}\}$ hold in case of Jacobi matrices with independent entries. The result can be easily extended to Wishart-type Jacobi matrices $\{J_N\}$ here by considering the filtration $\{\cF_k = \sigma\{c_i, d_i : i = 1,\dots, k\}\}_k$. The convergence of variances as in the previous corollary is one of the two sufficient conditions. The remaining one is quite similar to that in the Gaussian beta ensembles case, and hence, we will not mention it in details here. Consequently, the following CLTs for polynomial test functions follow.
\begin{theorem}
Let $p$ be a polynomial. Then the following hold.
\begin{itemize}
	\item[\rm(i)]	As $\beta N \to \infty$,
	\[
		\sqrt{\beta} N (\bra{L_N, p} - \Ex[\bra{L_N, p}] ) \dto \Normal(0, \sigma_p^2).
	\]
	\item[\rm(ii)]	As $\beta N \to 2c$,
	\[
		\sqrt{\beta} N (\bra{L_N, p} - \Ex[\bra{L_N, p}] ) \dto \Normal(0, \sigma_{p,c}^2).
	\]
\end{itemize}
\end{theorem}

\begin{remark}
\begin{itemize}
\item[(i)]
For fixed $\beta$, it was shown in \cite{DE06} that as $N \to \infty$,
\[
	N \bra{L_N, p} - N\bra{mp_\gamma, p} -  \left( \frac2\beta - 1\right)\bra{\mu_1, p}  \dto \Normal(0, \sigma_p^2/\beta),
\]
where $\mu_1$ is given by
\[
\mu_1=
		\frac14 \delta_{\lambda_{-}} +	\frac14 \delta_{\lambda_{+}} + \frac{1}{2\pi} \frac{1}{\sqrt{(\lambda_+ - x) (x - \lambda_-)}} \one_{(\lambda_-, \lambda_+) } (dx).
\]

\item[(ii)] Using results in case $\beta = 1,2$, we deduce that the limiting variance in the regime $\beta N \to \infty$ is given by (cf.~\cite[Theorem 7.3.1]{Pastur-book})
\[
	\sigma_f^2 = \frac{1}{2\pi^2} \int_{\gamma_-}^{\gamma^+}\int_{\gamma_-}^{\gamma^+} \left(\frac{f(y) - f(x)}{y - x}\right)^2 \frac{4\gamma - (x - \gamma_m)(y - \gamma_m)}{\sqrt{4 \gamma - (x - \gamma_m)^2} \sqrt{4 \gamma - (y - \gamma_m)^2}}dx dy,
\]
where $\gamma_m = (\gamma_{-} + \gamma_{+})/2 = (1 + \gamma)$.
\end{itemize}
\end{remark}

\subsection{$C^1$ test functions}

To extend CLTs from polynomial test functions to  $C^1$ test functions, one idea is to use a type of Poincar\'e inequality.  Consider (scaled) $\beta$-Laguerre ensembles with the joint density proportional to
\[
	 |\Delta(\lambda)|^\beta \prod_{i = 1}^N \left( \lambda_i^{\alpha} e^{-\eta\lambda_i} \right),
\]
where $\alpha = \frac{\beta}{2}(M - N + 1) - 1$, and $\eta > 0$. Directly from the joint density, we can derive a Poincar\'e inequality by using the following result. However, this approach requires $\alpha > 0$.
\begin{lemma}
[{\cite[Proposition~2.1]{Bobkov-Ledoux-2000}}]
\label{lem:Poincare}
Let $d\nu = e^{-V} dx$ be a probability measure supported on an open convex set $\Omega \subset \R^N$. Assume that $V$ is twice continuously differentiable and strictly convex on $\Omega$. Then for any locally Lipschitz function $F$ on $\Omega$,
\[
	\Var_\nu[F] = \int F^2 d\nu - \left(\int F d\nu \right)^2 \le \int (\nabla F)^t\Hess(V)^{-1} \nabla F d\nu.
\]
Here $\Hess(V)$ denotes the Hessian of $V$.
\end{lemma}

Let $\Omega = \{(\lambda_1, \dots, \lambda_N): 0 < \lambda_1 <\cdots < \lambda_N\} \subset \R^N$. Let $\nu$ be the distribution of the ordered eigenvalues of (scaled) $\beta$-Laguerre ensembles, that is, the probability measure on $\Omega$ of the form
\[
	d\nu = const \times |\Delta(\lambda)|^\beta \prod_{i = 1}^N \left( \lambda_i^{\alpha} e^{-\eta\lambda_i} \right) d\lambda_1 \cdots d\lambda_N= e^{-V} d\lambda_1 \cdots d\lambda_N,
\]
where
\[
	V = const + \eta\sum_{i = 1}^N \lambda_i - \alpha \sum_{i = 1}^N \log \lambda_i - \frac{\beta}{2} \sum_{i \neq j} \log|\lambda_j - \lambda_i|.
\]
Then the Hessian matrix of $V$ can be  easily calculated
\begin{align*}
	\frac{\partial^2 V}{\partial \lambda_i^2} &= \frac{\alpha}{\lambda_i^2} + \beta \sum_{j \neq i} \frac{1}{(\lambda_j - \lambda_i)^2},\\
	\frac{\partial^2 V}{\partial \lambda_i \partial \lambda_j} &= -\beta \frac{1}{(\lambda_j - \lambda_i)^2}.
\end{align*}
Observe that $\Hess (V) \ge D= \diag(\frac{\alpha}{\lambda_i^2})$. Here for two real symmetric matrices $A$ and $B$, the notation $A \ge B$ indicates that $A-B$ is positive semi-definite. It follows that $\Hess (V)^{-1} \le D^{-1}$. And hence, using Lemma~\ref{lem:Poincare} with
\[
	F = \frac{1}{N} \sum_{i = 1}^N f(\lambda_i),
\]
for a continuously differentiable function $f$,
we get the following inequality
\[
	\Var_\nu[F] \le \int ( \nabla F)^t\Hess(V)^{-1}\nabla F d\nu \le \int  (\nabla F)^t D^{-1}\nabla F d\nu.
\]
The inequality can be rewritten as
\begin{equation}
	\Var[\bra{L_N, f}] \le \frac{1}{\alpha N} \Ex[\bra{L_N, (\lambda f'(\lambda))^2}].
\end{equation}
This is one type of Poincar\'e inequality for $\beta$-Laguerre ensembles in this paper.

The restriction of the above inequality is that $\alpha$ must be positive, which does not hold in the regime $\beta N \to 2c$ when $c$ is small. The second approach based on the random Jacobi matrix model removes such restriction. We will end up with a slightly different inequality. Let us begin with several concepts.

A real random variable $X$ is said to satisfy a Poincar\'e inequality if there is a constant $c>0$ such that for any smooth function $f \colon \R \to \R$,
\[
	\Var[f(X)]  \le c \Ex[f'(X)^2].
\]
Here, by smooth we mean enough regularity so that the considering terms make sense. By definition, it is clear that $X$ satisfies a Poincar\'e inequality with a constant $c$, if and only if $\alpha X$ satisfies a Poincar\'e inequality with a constant $c\alpha^2$, for non-zero constant $\alpha$.

\begin{lemma}
	The chi distribution $\chi_k$ satisfies a Poincar\'e inequality with $c = 1$, that is,
	\[
		\Var[f(X)] \le \Ex[f'(X)^2], \quad X \sim \chi_k.
	\]
\end{lemma}
\begin{proof}
The probability density function of the chi distribution with $k$ degrees of freedom is given by
	\[
		\frac{1}{2^{\frac k2 - 1} \Gamma(\frac k2)} x^{k - 1} e^{-\frac{x^2}{2}}, \quad (x > 0).
	\]
Thus, for $k \ge 1$, the conclusion follows immediately from Lemma~\ref{lem:Poincare}.

	Next, we consider the case $0 < k <1$. Let $Y = X^{k}$. Then the probability density function of $Y$ is given by
	\[
		const\times \exp({-\frac{y^{\frac 2k}}{2}}),\quad ( y > 0).
	\]
By using Lemma~\ref{lem:Poincare} with $V = const + \frac{y^{\frac 2k}}2$, we obtain that
	\[
		\Var[g(Y)] \le \frac{k^2}{2-k}\Ex[Y^{2 - \frac 2k} g'(Y)^2].
	\]

For given $f(x)$, let $g(y) = f(y^\frac1k)$. Then
we see that
\[
	\Var[f(X)] = \Var[g(Y)] \le  \frac{k^2}{2-k}\Ex[Y^{2 - \frac 2k} g'(Y)^2] = \frac{1}{2-k}\Ex[f'(X)^2] \le \Ex[f'(X)^2].
\]
This means that $X$ satisfies a Poicar\'e inequality with a constant $c= 1$. The proof is complete.
\end{proof}

We need the following property.
\begin{lemma}[{\cite[Corollary 5.7]{Ledoux-book}}]
Assume that $X_i, (i = 1, \dots, k),$ satisfy Poicar\'e inequalities with constants $c_i$. Assume further that $\{X_i\}$ are independent. Then for any smooth function $g \colon \R^k \to \R$,
\[
	\Var[g(X_1, \dots, X_k)] \le (\max_i c_i)  \Ex[|\nabla g(X_1, \dots, X_k)|^2].
\]
\end{lemma}

Let $Y = (y_{mn})$ be an $M \times N$ real matrix, and $X = Y^t Y = (x_{ij})$. For a continuously differentiable function $f \colon \R \to \R$, let
\[
	g = g((y_{mn})) = \tr (f(X)).
\]
Then the partial derivatives of $g$ can be expressed as follows.

\begin{lemma}[cf.~{Eq.~(7.2.5) in \cite{Pastur-book}}]
It holds that
	\[
		\left( \frac{\partial g}{\partial y_{mn}}\right)_{M\times N} = 2Y f'(X).
	\]
Consequently
\[
	\sum_{m,n} \left( \frac{\partial g}{\partial y_{mn}}\right)^2 = 4\tr(Y f'(X) f'(X) Y^t ) = 4\tr(X f'^2(X)) = 4 \sum_{i = 1}^N \lambda_i f'(\lambda_i)^2.
\]
Here $\lambda_1, \dots, \lambda_N$ are the eigenvalues of $X$.
\end{lemma}

Combining three lemmas above, we arrive at another type of Poincar\'e inequality for $\beta$-Laguerre ensembles.
\begin{theorem}
Assume that $f$ is a continuously differentiable function. Then for $\beta$-Laguerre ensembles~\eqref{bLE}, the following inequality holds
\begin{equation}\label{Poincare-for-Laguerre}
	\Var[\bra{L_N, f}] \le \frac{4}{\beta M N} \Ex[\bra{L_N, x f'(x)^2}].
\end{equation}
\end{theorem}

The above Poincar\'e inequality is a key ingredient to  extend CLTs to $C^1$ test functions. Let us now prove Theorem~\ref{thm:CLT-full}.

\begin{proof}[Proof of Theorem~{\rm\ref{thm:CLT-full}}]
Assume that $f$ has continuous derivative $f'$ of polynomial growth. This implies that $f' \in L^2 ((1+x^2)d\nu_{\gamma, c}(x))$. Recall that for convenience,  $\nu_{\gamma, \infty}$ denotes the Marchenko--Pastur distribution with parameter $\gamma$.
We need the following property of measures determined by moments (called M.~Riesz's Theorem (1923) in \cite{Bakan-2001}): the measure $\mu$ is determined by moments, if and only if polynomials are dense in $L^2((1+x^2)d\mu(x))$. Consequently, there is a sequence of polynomials $\{p_k\}$ converging to $f'$ in $ L^2((1+x^2)d\nu_{\gamma,c}(x))$. It then follows that
\[
	\int x(p_k - f')^2 d\nu_{\gamma, c}(x) \le \frac12\int (p_k - f')^2 (1+x^2) d\nu_{\gamma, c}(x) \to 0 \quad \text{as}\quad k \to \infty.
\]

Let $P_k$ be a primitive of $p_k$. Since $P_k$ is a polynomial, for fixed $k$, as $N \to \infty$,
\begin{equation}\label{CLT-XNk}
	X_{N, k}:= \sqrt{\beta} N (\bra{L_N, P_k} - \Ex[\bra{L_N, P_k}]) \dto \Normal(0, \sigma_{P_k, c}^2), \quad \Var[X_{N, k}] \to  \sigma_{P_k, c}^2.
\end{equation}
Let $Y_N = \sqrt{\beta} N (\bra{L_N, f} - \Ex[\bra{L_N, f}])$. Then by the Poincar\'e inequality~\eqref{Poincare-for-Laguerre},
\begin{equation}\label{Variance-approx}
	\Var[ Y_N - X_{N, k} ] \le \frac{4N}{ M} \Ex[\bra{L_N, x(f' - p_k)^2}],
\end{equation}
which implies that
\[
	\lim_{k \to \infty} \limsup_{N \to \infty} \Var[ Y_N - X_{N, k} ] \le 4 \gamma  \lim_{k \to \infty} \bra{\nu_{\gamma, c}, x(f' - p_k)^2} = 0.
\]
Here we have used the property that for continuous function $g$ of polynomial growth,
\[
	\Ex[\bra{L_N, g}] \to \bra{\nu_{\gamma, c}, g}.
\]
Then, the CLT for $Y_N$ follows from the equations \eqref{CLT-XNk} and \eqref{Variance-approx} by using the following general result whose proof can be found in \cite{Nakano-Trinh-2018} or \cite{Trinh-2018-CLT}. The proof is complete.
\end{proof}
\begin{lemma}\label{lem:CLT-triangle}
Let $\{Y_N\}_{N=1}^\infty$ and $\{X_{N,k}\}_{N,k=1}^\infty$ be mean zero real random variables. Assume that
\begin{itemize}
	\item[\rm(i)] for each $k$, as $N \to \infty$,
		$	X_{N,k} \dto \Normal(0, \sigma_k^2),$ and $\Var[X_{N, k}] \to \sigma_k^2$;
	\item[\rm(ii)]
		$
			\lim_{k \to \infty} \limsup_{N \to \infty} \Var[X_{N,k} - Y_N] =0.
		$
\end{itemize}
Then the limit $\sigma^2 = \lim_{k \to \infty} \sigma_k^2$ exists, and as $N \to \infty$,
\[
 	Y_N \dto \Normal(0, \sigma^2), \quad \Var[Y_N] \to \sigma^2.
\]
\end{lemma}

\bigskip
\noindent{\bf Acknowledgment.}
This work is supported by the VNU University of Science under project number TN.18.03 (H.D.T) and by JSPS KAKENHI Grant Number JP19K14547 (K.D.T.).


Visibility-preserving convexifications using single-vertex moves

Abstract: Devadoss asked: (1) can every polygon be convexified so that no internal visibility (between vertices) is lost in the process? Moreover, (2) does such a convexification exist, in which exactly one vertex is moved at a time (that is, using {\em single-vertex moves})? We prove the redundancy of the "single-vertex moves" condition: an affirmative answer to (1) implies an affirmative answer to (2). Since Aichholzer et al. recently proved (1), this settles (2).

\section{Introduction}

The problem of {\em convexifying} a polygon (that is, continuously
transforming it, while maintaining simplicity) is a classical,
well-studied problem in computational geometry. Different
restrictions on the transformations allowed give rise to several
versions of this problem. In the most famous variant,
the lengths of the edges are to be
maintained; this is the celebrated Carpenter's Rule Theorem
(see~\cite{cdr,Stre}).

Devadoss~\cite{demlan} proposed another variant: (1)
can every polygon be convexified
without losing internal visibility between any pair of
vertices? (equivalently: does every polygon admit a {\em
  visibility-preserving} convexification?)
 Devadoss~\cite{devadoss} also raised a stronger version of this
 question: (2) does there exist a
 visibility-preserving convexification
in which at most one vertex is moved at a time (that
is, by {\em single-vertex moves})? Partial answers to (1) and (2) were given in~\cite{acflsu} and~\cite{devadoss}.

Our main result is the following.

\begin{theorem}\label{thm:maintheorem}
An affirmative answer to (1) implies an affirmative answer to
(2). That is: if every polygon has a visibility-preserving
convexification, then every polygon has a visibility-preserving convexification using only
single-vertex moves.
\end{theorem}

Since (1) has been recently settled in the afirmative by Aichholzer et
al.~\cite{aetal}, we have the following.

\begin{corollary}
Every polygon has a visibility-preserving convexification using only
single-vertex moves. \hfill$\Box$
\end{corollary}

In Section~\ref{sec:general} we formalize the relevant notions at play
 (such as visibility-preserving transformations and single-vertex
moves).
The proof of Theorem~\ref{thm:maintheorem} is in
Section~\ref{sec:proofmain}.

\section{Visibility, transformations, and
  single-vertex moves}\label{sec:general}

A {\em polygon} consists of a finite set $\{p_1, p_2,
\ldots, p_n\}$ of points (the {\em vertices}), plus the straight segments (the
      {\em edges}) joining $p_i$ to $p_{i+1}$ for $i=1,2,\ldots,n$
      (indices are read modulo $n$).
Throughout this work we implicitly assume
      that all polygons under consideration are {\em simple}: edges
      only intersect in a common endpoint.
Two points
in $P$ are {\em internally visible} (for brevity, {\em visible}) in $P$
if the open segment
joining them is contained in the
interior of $P$. We also say that the points are $P$-{\em visible}.
If $p,q$ are $P$-visible vertices, then $\{p,q\}$ is a
$P$-{\em visible pair}.

To speak meaningfully about the continuity of the transformations
under consideration, first we must agree on the underlying metric. Let
$\eucl{p}{q}$ denote the usual Euclidean distance between points $p$ and $q$.  Let
$P,Q$ be polygons, and let $p_1, p_2,
\ldots, p_n$ and $q_1, q_2,\ldots, q_n$ be their respective vertices
(distance is only defined between polygons with the same number of vertices).
The  {\em
  distance} $d(P,Q)$ between $P$ and $Q$ is $\min_\sigma\{ \max_{i\in\{1,2,\ldots,n\}}\{\eucl{p_i}{
q_{\sigma(i)}} \}  \}$, where the minimum
is taken over all bijections $\sigma:\{1,2,\ldots,n\}\to \{1,2,\ldots,n\}$.

Let
${\mathscr P}_n$ denote the collection of all polygons on $n$ vertices.
If $P\in{\mathscr P}_n$ and $\delta > 0$, we let
$N_\delta(P)$ denote the set of polygons $Q\in{\mathscr P}_n$ such that $d(P,Q)
< \delta$.

The polygonal transformations under consideration are obtained by
moving the vertices of the polygon; their incident edges get dragged
along.  Thus, a transformation on a polygon $P$ gets induced by defining, for each
vertex $p$ of $P$, a (time-parameterized) mapping that follows the
movement of
$p$.  The time domain is any closed interval $[a,d]$, and the {\em
  orbit} of $p$ is a continuous mapping $t\mapsto p^t$ such that $p^a
= p$.
Thus, for each $t\in [a,d]$ we
obtain a polygon $P^t$ (where $P^a = P$), and the transformation
(which we denote $P^{[a,d]}$) is a continuous mapping from $[a,d]$ to
${\mathscr P}_n$, transforming the
{\em initial} polygon $P^a =P$ onto the {\em final} polygon $P^d$.
If $P^d$ is convex, then the transformation is a {\em
  convexification} of $P^a=P$.
If only one orbit is not the constant map, then the
transformation is a {\em single-vertex move}.

Now let $P\in {\mathscr P}_n$ be a polygon with vertices $p_1, p_2, \ldots, p_n$. A
transformation ${\{P^t\ | \ t\in [a,b]\}}{a}{d}$ on $P$ is
{\em visibility-preserving} if for all $i,j\in
\{1,2,\ldots,n\}$ and all $a \le s \le t \le d$, if $p^s_i$ and
$p^s_j$ are $P^s$-visible,  then $p^{t}_i$ and $p^{t}_j$ are
$P^{t}$-visible. If in addition there exist $i,j\in \{1,2,$ $\ldots,n\}$ such that $p^a_i$ is not
  $P^a$-visible from $p^a_j$, but $p^d_i$ is $P^d$-visible from
  $p^d_j$, then the transformation is {\em visibility-increasing}.

Let $k\ge 3$. A $k$-tuple $(q_1, q_2, \ldots, q_k)$ of vertices of a polygon $P$ is {\em critical} if
there is a straight segment that contains $q_1, q_2, \ldots, q_k$, and does not
contain any point in the exterior of $P$. A polygon is {\em
  critical} if it contains a critical $k$-tuple for some $k \ge 3$.
The proof of the following is a trivial exercise.

\begin{observation}\label{obs:sta}
If $P$ is a critical polygon, then
there is a visibility-increasing single-vertex move with
initial polygon $P$.\hfill$\Box$
\end{observation}

Let $P$ be a polygon, and let $\delta>0$ be small enough so that for
every $Q\in N_\delta(P)$, there is a unique natural bijection between
the vertices of $P$ and the vertices of $Q$. We say that
$\delta$ is a {\em safe radius} for $P$ if for every $Q\in N_\delta(P)$,
a triple of vertices of $Q$ is critical only if the corresponding triple in $P$ is
also critical.  The proof of the following is straightforward.

\begin{observation}\label{obs:safe}
Every polygon has a safe radius.\hfill$\Box$
\end{observation}

\section{Proof of Theorem~\ref{thm:maintheorem}}\label{sec:proofmain}

Let $P\in{\mathscr P}_n$, and let $p_1, p_2, \ldots, p_n$ be the vertices of $P$.
With $n$ fixed, we proceed by induction on the number of nonvisible pairs. Since
adjacent vertices are not visible to each other, it follows that the
minimum number of nonvisible pairs is $n$.  If there
are exactly $n$ nonvisible pairs, then $P$ is convex, and there is nothing to
prove. Thus we assume that $P$ has $m>n$ nonvisible
pairs, and that the result holds for every polygon in ${\mathscr P}_n$
with fewer than $m$ nonvisible pairs.

By assumption, there exists a visibility-preserving
convexification ${\{P^t\ | \ t\in [a,b]\}}{a}{d}$ of $P$.
An elementary real
analysis argument shows that there exists a
  smallest $c\in[a,d]$ such that $P^c$ is critical. We note that
  if $c=a$, then by Observation~\ref{obs:sta}
we are done, using
  the induction hypothesis. Thus we may assume that $c > a$.

Since $t\mapsto P^t$ is continuous for all $t\in[a,c]$,
Observation~\ref{obs:safe} and the compactness of
$[a,c]$ imply that there is a $\delta>0$ that is a safe radius for $P^t$ for
every $t\in[a,c]$.
A similar elementary continuity argument using the compactness of
$[a,c]$ shows that there exists a $\tau>0$ such that, for all
$s,t\in[a,c]$ such that $|s-t|<\tau$, $d(P^{s},P^{t})<\delta$.

Evidently, we may choose $\tau$
so that there is an integer $L$ such that $c-a=L\tau$.

Let $b:=c-\tau$. We claim that for $\ell=0,1,\ldots, L-2$ there is a sequence of
visibility-preserving single-vertex moves that takes $P^{a+\ell\tau}$ to
$P^{a+(\ell+1)\tau}$. Indeed, it suffices to move (in any
order), for
$i=1,2,\ldots,n$,
$p_i^{a+\ell\tau}$ to $p_i^{a+(\ell+1)\tau}$ following the orbit $p_i^{[a+\ell\tau,a+(\ell+1)\tau]}$.
The choice of $\tau$ guarantees that each of these moves is
visibility-preserving.  We conclude that there is a sequence of
visibility-preserving single-vertex moves that takes $P^a$ to $P^b$.

Once we reach $P^{b}$ we proceed similarly towards $P^c$, moving each
vertex $p^b_i$ towards $p^c_i$, one at a time, for $i=1,2,\ldots,n$.
Consider the first critical polygon $Q$ reached in this last stage
(such a $Q$ obviously exists, since $P^c$ itself is
critical). Visibility losses (respectively, gains) may occur only when
(respectively, immediately after) reaching critical polygons.  Thus,
the choice of $Q$ implies that before reaching $Q$, we cannot have
lost visibility.

Suppose first that in the moment we reach $Q$ we lose visibility; we
shall derive a contradiction.
In this case, the moving vertex (we recall we reach $Q$ via a
single-vertex move) must be part of a critical triple at
$Q$, and two of these vertices must lose visibility at the moment of
reaching $Q$. Immediately before reaching $Q$, the corresponding three
vertices clearly satisfy the following (which we call {\em Property A}): each two of them are either a visible
pair or adjacent. Since before reaching $Q$
there is no loss of visibility, the corresponding three vertices in
$P^b$ (and hence all the way back to $P^a$) must also satisfy Property A.
But the corresponding three vertices in $P^c$ cannot satisfy
Property A: indeed, the choice of $\delta$ and $\tau$, plus the fact that they
are a critical triple in $Q$, imply that they
must form a critical triple in $P^c$, but no critical triple satisfies
Property A. This implies that there is a pair of vertices that are
$P^b$-visible but not $P^c$-visible, contradicting that ${\{P^t\ | \ t\in [a,b]\}}{a}{c}$ is
visibility-preserving.

Suppose finally that at the moment of reaching $Q$ we do not
lose visibility.  In this case, we have then reached $Q$ from
$P^b$ (and hence from $P^a$ as well) by a sequence of
visibility-preserving single-vertex moves. Since $Q$ is critical, then
we are done by Observation~\ref{obs:sta} and the
induction hypothesis.
\hfill$\Box$

Computational Geometry (cs.CG); Robotics (cs.RO); Combinatorics (math.CO)