Efficient Concentration with Gaussian Approximation

Abstract: Concentration inequalities for the sample mean, like those due to Bernstein and Hoeffding, are valid for any sample size but overly conservative, yielding confidence intervals that are unnecessarily wide. The central limit theorem (CLT) provides asymptotic confidence intervals with optimal width, but these are invalid for all sample sizes. To resolve this tension, we develop new computable concentration inequalities with asymptotically optimal size, finite-sample validity, and sub-Gaussian decay. These bounds enable the construction of efficient confidence intervals with correct coverage for any sample size. We derive our inequalities by tightly bounding the Hellinger distance, Stein discrepancy, non-uniform Kolmogorov distance, and Wasserstein distance to a Gaussian, and, as a byproduct, we obtain the first explicit bounds for the Hellinger CLT.

\section{Introduction}
Concentration inequalities for the sample mean are a staple of probability theory, statistics, and machine learning. Given $n$ independent and identically distributed (\textrm{i.i.d.}\xspace) observations  $W_1,\dots,W_n$ they allow us to give finite sample and high probability guarantees that the sample mean $\bar{W}_n\defeq \frac{1}{n}\sum_{i=1}^n W_i$ is not too far away from the population mean $\mathbb{E}(W_1)$. Specifically, they provide upper bounds for the probability $P\big(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n}\big)$ for each $t \ge 0$.
Such concentration inequalities lie at the heart of decision-making in reinforcement learning \citep{mnih2008empirical,audibert2009exploration},
generalization guarantees in high-dimensional statistics, machine learning, and deep learning \citep{wainwright2019high,bartlett2002rademacher,zhou2018non}, and
the design of efficient learning procedures \citep{maurer2009empirical}.

However, standard concentration inequalities are overly conservative yielding confidence intervals that are unnecessarily wide and generalization guarantees that are weaker than needed. This is notably the case for the commonly used concentration inequalities of \citet{hoeffding1963probability} and \citet{Bernstein46}. For random variables with  bounded deviations $|W_1 - \E(W_1)| \leq R$ and variance $\sigma^2\defeq \Var(W_1)$, the Hoeffding  and Bernstein inequalities respectively state that
\begin{align}\label{eqhoeffberntails}
P\big(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n}\big)\le e^{-\frac{t^2}{2R^2}}
\qtext{and}
P\big(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n}\big)\le e^{-\frac{t^2}{2(\sigma^2+Rt/3)}}.
\end{align}
Meanwhile, the central limit theorem (CLT) identifies the exact limit for these tail probabilities:
\begin{align}\label{eq:gaussian_tails}
P\big(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n}\big)\xrightarrow{n\rightarrow \infty}\Phi^c\Big(\frac{t}{\sigma}\Big)
\end{align}
where $\Phi(\cdot)$ is the cumulative distribution function (CDF) of a standard normal distribution and  $\Phi^c(\cdot)=1-\Phi(\cdot)$.
As a result, standard confidence intervals based on the CLT are asymptotically exact and often much narrower than those obtained using concentration inequalities. However, these intervals are typically only asymptotically valid and provide incorrect coverage for every sample size $n$.

The choice between concentration inequalities that are finite-sample valid but loose and CLT-based approximations that are asymptotically tight but invalid is very unsatisfying. In this paper we derive new bounds that offer the best of both worlds: our new concentration inequalities are both finite-sample valid and \emph{efficient}---that is, asymptotically of minimal width when scaled by $\sqrt{n}$. We achieve this by deriving explicit and practical bounds for the distance between the sample mean
and a Gaussian with matching variance.
In \cref{hell_sec,stein_sec,simple}, we measure the distance in three different ways: via
the Hellinger distance, the Stein discrepancy (as defined by \citet{ledoux2015stein}), and the $p$-Wassertein distance,  respectively.
In each case, our Gaussian approximation bounds yield efficient concentration inequalities of the form
\begin{align}\label{eq_intro}\P(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n})\le \Phi^c(t/\sigma)+r_n(t/\sigma)
\end{align}
with explicit remainder terms $r_n(t/\sigma)$.
For example, in the Hellinger case, $r_n(t/\sigma)$ decreases at an $O(\log(n)/\sqrt{n})$ rate in $n$ and at a sub-Gaussian $e^{-\Omega(t^2)}$ rate in $t$.

It is informative to compare these results with those obtained using classical CLT corrections.
For random variables with bounded deviations,
the Berry-Esseen bound \citep{esseen1942liapunov} guarantees that
\begin{align}\label{berryesseen}
|\P(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n})-\Phi^c(t/ \sigma )|
\le
\frac{CR}{\sigma\sqrt{n}}
 \qtext{for all} t \geq 0
\end{align}
and some universal constant $C$.
This yields an efficient concentration inequality, but the bound is overly conservative as the correction is independent of $t$. Non-uniform Berry-Esseen bounds  \citep{nagaev1965some,bikyalis1966estimates} ameliorate this behavior by
identifying a constant $\tilde C$ satisfying $$|P\big(\bar{W}_n-\mathbb{E}(W_1)\ge t/\sqrt{n} \big)-\Phi^c\big(t/\sigma\big)|
\le
\frac{\tilde C R}{\sigma\sqrt{n}(1+t/\sigma)^3} \qtext{for all} t \geq 0.$$
Appealingly, this non-uniformity yields tighter bounds for larger $t$.
However, the correction has only cubic, that is, $O(t^{-3})$, decay in $t$ as the underlying argument only exploits the existence of a third moment of $\bar{W}_n$.
Quantile coupling inequalities \citep[see, e.g.,][]{mason2012quantile} like the groundbreaking Komlós-Major-Tusnády approximations \citep[Thm.~1]{komlos1975approximation,komlos1976approximation,bretagnolle1989hungarian} and the strong embedding bounds of \citet{chatterjee2012new,bhattacharjee2016strong} improve this $t$ dependence for $W_1$ with finite exponential moments but provide at best exponential decay.
By exploiting the sub-Gaussianity of $W_1$, our theorems provide a correction term with \emph{squared-exponential}  $e^{-\Omega(t^2)}$ decay, leading to much sharper bounds for larger $t$.

As the Hellinger, Stein, and Wasserstein distances each measure separate properties of the empirical average, the inequalities we obtain are complementary and hold under distinct conditions.
For example, when controlling the Hellinger distance, the remainder term in \cref{eq_intro} depends on the score function of the random variable $W_1$.
When controlling the Stein discrepancy, the remainder term instead depends on the moments of a quantity called the \emph{Stein kernel}, defined in \cref{stein_sec}.
In this setting, we derive an improved $O(\sqrt{\log (n) / n})$ rate of decay in $n$ on top of the sub-Gaussian decay in $t$.
As the Stein kernel and the score function can be controlled in different settings those bounds complement each other.
Finally, the bounds obtained using $p$-Wassertein distances apply to \emph{all} bounded-deviation variables with no additional distributional assumptions, at the cost of somewhat larger remainder terms. We note in general that when the random variables are continuous and the score function can be controlled (see \cref{bound} and \cref{stein_thm_3}) we can improve the tightness of our results by avoiding a lossy union bound. This is complemented by tighter efficient bounds for smoothed observations, which will hold under minimum conditions.
We also propose new quantile bounds that are also valid for finite samples {and} efficient. Finally, with analogy to the empirical Bernstein bound \citep{mnih2008empirical,audibert2009exploration,maurer2009empirical}, we also develop new \emph{empirical Berry-Esseen bounds} that are efficient and finite-sample valid even when the variance parameter $\sigma^2$ is unknown. We confirm numerically that these new bounds improve upon the most commonly used confidence intervals and the efficient $I_{n,3}$ confidence interval of \citet{romano2000finite}.

We note that as a by-product of our inquiry we obtain, for the first time, an explicit rate of convergence for the central limit theorem in terms of the Hellinger distance, which is of independent interest.  The bound obtained is computable for random variables $(W_i)$ whose score functions admit finite moments. This is to contrast with the currently known rates of convergence of the entropic central limit theorems which are either not in closed form \citep{bobkov2013rate}, require conditions on the spectral gap \citep{artstein2004rate}, or depend on the Stein kernel \citep{ledoux2015stein}. In addition, we propose a novel bound that, beyond empirical averages, upper-bounds the Hellinger distance between general random variables and a normal. %
\subsection{Overview}
In \cref{def} we define the notion of efficient concentration and quantiles, present some of the assumptions we will work with, and introduce notations. In \cref{hell_sec} we propose efficient concentration inequalities using the Hellinger distance, under the condition that the random variables are continuous. Moreover, we propose the first-ever computable rate of convergence of the CLT in terms of the Hellinger distance. In \cref{stein_sec} we propose three different efficient tail bounds under the condition that the random variables $Y_1$ admits a Stein kernel. Each one of those tail bounds will hold under different conditions and we contrast those to each other and previously known techniques. In \cref{simple} we use the $p$-Wassertein distances to obtain efficient tail bounds and efficient quantiles under the minimum amount of conditions. In \cref{smoothed} we obtain even tighter efficient concentration inequalities for smoothed observation. Finally, in \cref{empi} we propose empirical efficient quantile bounds that can be computed without requiring any prior knowledge about the variance and present some numerical evaluations of our bounds. In the different sections, we also present sketches of the proofs of the derived results.
\subsection{Notation}\label{sec:notation}
Throughout, we will let $\varphi(x)\defeq\frac{1}{\sqrt{2\pi}}e^{-x^2/2}$ designate the probability density function (PDF) of a standard normal random variable and write $a_n\lessapprox b_n$ to indicate that two sequences $(a_n)$ and $(b_n)$ satisy $a_n\le b_n[1+o_n(1)]$.  We will also make use of the following notation:$$Y_i\defeq W_i-\E(W_1),\qquad S_n\defeq\sqrt{n} (\bar{W}_n  - \E(W_1)),\qquad  \tilde S_n\defeq\sigma^{-1}S_n, \qquad \tilde R\defeq\sigma^{-1}R,$$
$$A_p\defeq \frac{\sqrt{e}\sqrt{p+2}(2e)^{1/p}}{\sqrt{2}},\qquad  A^*_{n,p}\defeq \frac{({p+2})n^{1/p}}{2\sqrt{n}},\qquad \tilde A_{n,p}\defeq A^*_{n,p}\tilde R^{-2/p}.$$

\section{Defining Efficient Concentration}\label{def}
To match the setting of the classical Bernstein inequality \eqref{eqhoeffberntails}, we will focus on random variables satisfying the following bounded deviations assumption.
\def$(\tau_{\tilde Y_1},p')${$(R,\sigma)$}
\addtocounter{assumption}{-1}
\begin{assumption}\label{assump:bounded-dev}
The scaled deviations $(S_n)_{n\geq 1}$ satisfy $S_n = \sqrt{n} (\bar{W}_n - \E(W_1))$ for a shared sequence of \textrm{i.i.d.}\xspace~variables $(W_i)_{i\geq 1}$ with $\bar{W}_n\defeq \frac{1}{n}\sum_{i=1}^n W_i$. We write the centered random variables $Y_i\defeq W_i-\mathbb{E}(W_1)$, and we suppose that $\Var(Y_1) = \sigma^2$, and $|Y_1|\leq R$ almost surely.
\end{assumption}

Our first inferential goal is to tightly upper bound the tail probability $\P(S_n \geq t)$ for a given threshold $t \geq 0$.
The CLT provides an asymptotic lower bound for this problem as $\P(S_n \geq t)$ is known to converge precisely to $\Phi^c(\frac{t}{\sigma})$ as $n$ increases.
\begin{prop}[Asymptotic lower bound for valid tail bounds]
\label{tail_lower_bound}
Fix any $t \geq 0$, any sequence of scaled deviations $(S_n)_{n\geq 1}$ satisfying \cref{assump:bounded-dev}, and any sequence of candidate tail bounds $(\delta_n(t))_{n\geq 1}$.
If $\P(S_n \geq t) \leq \delta_n(t)$ for all $n$, then $\Phi^c(\frac{t}{\sigma}) \leq \liminf_{n\to\infty} \delta_n(t)$.
\end{prop}
\begin{proof}
Suppose that a sequence $\delta_n(t)$ satisfies  $\liminf_{n\to\infty} \delta_n(t) < \Phi^c(\frac{t}{\sigma})$.  Then there exists an $\epsilon > 0$ such that $\delta_n(t) \leq \Phi^c(\frac{t}{\sigma}) - \epsilon$ for infinitely many $n$.
However, by the CLT \citep[see, e.g.,][Thm.~3.4.1]{durrett2019probability}, there exists an $n_{\epsilon}$ such that, for all $n > n_{\epsilon}$, $\Phi^c(\frac{t}{\sigma}) - \epsilon < \P(S_n \geq t)$.  Therefore, $\delta_n(t) < \P(S_n \geq t)$ for infinitely many $n$, confirming the claim via its contrapositive.
\end{proof}

Unfortunately, the CLT limit does not provide a suitable tail bound for any finite $n$, as $\P(S_n \geq t) > \Phi^c(\frac{t}{\sigma})$ for many $S_n$ satisfying \cref{assump:bounded-dev}. However, by tightly bounding the distance between the distribution of $S_n$ and the distribution of a Gaussian we can correct the asymptotic bound to obtain one that is both valid in finite samples and asymptotically exact.
We will call such bounds \emph{efficient concentration inequalities}.

Our second inferential goal is to tightly bound the quantiles of $S_n$.
That is, given a tail probability $\delta \in (0,1)$ we wish to find $q_n(R,\delta,\sigma)$ (a measurable function of $(W_i)_{i=1}^n, R,\sigma,$ and $\delta$) satisfying $\P(S_n\ge q_n(R,\delta,\sigma))\le \delta$. Such quantile bounds immediately deliver both one- and two-sided confidence intervals for the population mean $\E(W_1)$ as
\begin{talign}
\P(\bar{W}_n - \frac{q_n(R,\delta,\sigma)}{\sqrt{n}} \leq \E(W_1)) \geq 1-\delta
\qtext{and}
\P(|\E(W_1) - \bar{W}_n| \leq \frac{q_n(R,\delta/2,\sigma)}{\sqrt{n}}) \geq 1-\delta.
\end{talign}
The interval efficiency theory of
 \citet{romano2000finite} implies that the CLT once again provides an asymptotic lower bound for any valid sequence of quantile bounds.
\begin{prop}[Asymptotic lower bound for valid quantile bounds]
\label{quantile_lower_bound}
Fix any $R > 0$, any $\delta \in (0,\frac{1}{2})$, and any nonnegative candidate quantile bounds $(q_n(R,\delta,\sigma))_{n\geq 1, \sigma >0}$.
Suppose that, for each $\sigma \in (0,R]$ and all $(S_n)_{n\geq 1}$ satisfying \cref{assump:bounded-dev},
$\P(S_n \geq q_n(R,\delta,\sigma)) \leq \delta$
for all $n$.
Then $(q_n(R,\delta,\sigma))_{n\geq 1}$ is not asymptotically concentrated\footnote{A sequence of nonnegative random variables $(X_n)_{n\geq 1}$ is asymptotically concentrated on $[0,a]$ if $(X_n - a)_+ \stackrel{p}{\to} 0$.} on $[0,a]$ for any $a < -\sigma \Phi^{-1}(\delta)$.
\end{prop}

\begin{proof}
The result follows by applying Thm.~2.1 of  \citet{romano2000finite} to the conservative  confidence intervals $I_n(\delta, \sigma) = \bar{W}_n + \frac{1}{\sqrt{n}}[-q_n(R,\delta,\sigma), q_n(R,\delta, \sigma)]$ for the unknown mean $\E(W_1)$.
\end{proof}
To construct \emph{efficient quantile bounds}, we will once again tightly bound the distance between $S_n$ and its Gaussian limit.

\section{Efficient Concentration with Hellinger Approximation}\label{hell_sec}
Let $X$ and $Y$ be two continuous real-valued random variables with respective probability density functions (PDFs) $f_X$ and $f_Y$ then the Hellinger distance is defined as $$H(X,Y)^2\defeq\frac{1}{2}\int_{\mathbb{R}}\big(\sqrt{f_X(x)}-\sqrt{f_Y(x)}\big)^2dx.$$In this section we show how the Hellinger distance can be used to obtain efficient concentration inequalities, and derive a bound for the Hellinger distance between $S_n$ and its normal limit. Note that by doing so we establish, for the first time, an explicit rate of convergence for the central limit theorem in terms of the Hellinger distance, which is of independent interest.

\noindent Throughout this section, we suppose that the random variables $(Y_i)$ are real valued continuous random variables. %

\def$(\tau_{\tilde Y_1},p')${$(\tilde f)$}
\addtocounter{assumption}{-1}
\begin{assumption}
\label{score_function_existance}The random variable $Y_1$ is continuous with a probability density function $\tilde f$ that is continuously differentiable almost everywhere on its support. %
\end{assumption}

\subsection{Concentration inequalities and the Hellinger distance} Under \cref{score_function_existance} the Hellinger distance between $S_n$ and its normal limit is properly defined and upper bounded by $2$. We show that this can be used to obtain efficient concentration inequalities for $S_n$ if the assumption \cref{assump:bounded-dev}  also holds.
In this goal, we take $\hat{Q}_n(u)$ to be any known  upper bound for $P\big(S_n\ge u\sigma\big)$ and $\hat{Q}^d_n(u)$ to be any known  upper bound for $P\big(|S_n|\ge u\sigma\big)$. Possible choices are the classical Hoeffding or Bernstein bounds \cref{eqhoeffberntails} or  the new Wasserstein-approximation bounds we derive in \cref{simple} (which hold whenever \cref{assump:bounded-dev} holds).

\begin{theorem}[Efficient Hellinger tail bound]\label{eff_hellinger} Suppose that the random variables $(Y_i)$ satisfy \cref{assump:bounded-dev} and \cref{score_function_existance}. Let $\hat{Q}_n(u)$ be an any arbitrary upper bound to $\P(S_n\ge \sigma u)\le \hat{Q}_n(u)$. We denote by $H(\cdot,\cdot)$ the Hellinger distance and write $Z\sim N(0,1)$ as a standard normal random variable. Then the following holds
\begin{align}
 \P(S_n\ge \sigma u)\le\Phi^c(u)+ \sqrt{2}H(\sigma ^{-1}S_n,Z)\Big[\sqrt{\hat Q_n(u)}+\sqrt{\Phi^c(u)}\Big]
\end{align}Moreover let $\hat{Q}^d_n(u)$ be an any arbitrary upper bound to $\P(|S_n|\ge \sigma u)\le \hat{Q}^d_n(u)$
\begin{align}
P\big(|S_n|\ge \sigma u\big)
            &\le 2\Phi^c(u)+\sqrt{2}H(\sigma^{-1}S_n,Z)\Big[\sqrt{2\Phi^c(u)}+\sqrt{\hat Q_n^d(u)}\Big]
\end{align}\end{theorem}
Remark that in \cref{bound} we prove that there is a constant $K$, that we derive, such that $\sqrt{2}H(\sigma^{-1}S_n,Z)\le \frac{K\log(n)}{\sqrt{n}}$. This directly implies that: $$\P(S_n\ge \sigma u)\le \Phi^c(u)+\frac{K\log(n)}{\sqrt{n}}\Big[\sqrt{\hat Q_n(u)}+\sqrt{\Phi^c(u)}\Big]\le \Phi^c(u)+\frac{K\log(n)}{\sqrt{n}}\Big[\sqrt{e^{-\frac{u^2\sigma^2}{2R^2}}}+\sqrt{\Phi^c(u)}\Big].$$We note that in practice we will want to use the upper bound derived in
\cref{cpt_concentration} for $\hat{Q}_n(u)$ rather than the Hoeffding bound which will provide us with significantly tighter upper bounds. We also remark that the correction term $\frac{K\log(n)}{\sqrt{n}}\Big[\sqrt{e^{-\frac{u^2\sigma^2}{2R^2}}}+\sqrt{\Phi^c(u)}\Big]$ is sub-Gaussian in $u$ and decreases as $\frac{\log(n)}{\sqrt{n}}.$

\subsection{CLT rate of convergence in terms of the Hellinger distance} In this section we derive the rate of convergence of $S_n$ to its normal limit in terms of the Hellinger distance which is of independent interest as no explicit bound was previously known. To do so we need to impose some moment conditions on the score function of $Y_1$ which we denote $S_{Y_1}(\cdot)$ and is defined as $S_{Y_1}(\cdot)=-\frac{\tilde f'(\cdot)}{\tilde f(\cdot)}.$%

\def$(\tau_{\tilde Y_1},p')${$(S_{Y_1},p')$}
\addtocounter{assumption}{-1}
\begin{assumption}
    \label{moment_score} Let $p'\ge 1$, we assume that the norm $\Big\|S_{Y_1}(\sigma Y_1)\Big\|_{\frac{2p'}{p'-1}}<\infty$ is finite. Write $\Omega\subset\mathbb{R}$ the support of $Y_1$, we suppose that $\tilde f\big|_{\delta \Omega}=0$.%
\end{assumption} Let $Z\sim N(0,1)$ be a standard normal random variable. For all $p\in [1,p']$, we introduce the following notation for $A_{n,\frac{2p}{p-1}}$ and $A^*_{n,\frac{2p}{p-1}}$ defined in \cref{sec:notation}:
$$C_p\defeq\min\begin{cases}
    \sqrt{\frac{2p}{p-1}-1}\|S_{Y_1}(\sigma Y_1)\|_{\frac{2p}{p-1}}
    \\    A^*_{n,\frac{2p}{p-1}}\|S_{Y_1}(\sigma Y_1)\|_{\frac{2p}{p-1}}+  A_{n,\frac{2p}{p-1}}\|S_{Y_1}(\sigma Y_1)\|_{2}
\end{cases}.$$
\begin{theorem}[Hellinger CLT bound]\label{bound}Let $p'\ge 1$ be a real. Suppose that the random variables $(Y_i)$ satisfy \cref{assump:bounded-dev}, \cref{score_function_existance} and \cref{moment_score}. Write $\tilde R\defeq\frac{R}{\sigma}$. Then we obtain that
   \begin{align}
 \sqrt{2}  H(\tilde S_n,Z)
   &\lessapprox \inf_{\substack{D>0\\\kappa\ge \frac{\tilde R^2}{\sqrt{n}D}}}\Big\{\frac{{\sqrt{e}\tilde R^2}(e^{2\kappa}-1)}{\sqrt{3}D\sqrt{n}\kappa }+\frac{\log(n/D^2)}{4\sqrt{2}\sqrt{n}} (\tilde R-1)-\frac{1}{2}\log(1-\frac{D}{\sqrt{n}})\\&\qquad\qquad+\inf_{p'\ge p\ge 1}C_p\Big\{\frac{\sqrt{2p-1}\tilde R}{\sqrt{n\kappa}}+\frac{2\sqrt{e}\sqrt{2p-1}\tilde R}{\sqrt{3n\kappa}}\Big[e^{{2(2p-1)\kappa}}-1-{2(2p-1)\kappa}\Big]\Big[2-\frac{2}{3\tilde R}\Big]
\\&\qquad\qquad +\frac{\tilde R^2}{2n\kappa M_{n,\kappa}} \Big[U_{n,2p}
 +\tilde U_{n,2p}\Big]\Big\}\Big\}
\end{align}
Moreover they are constants $K_1$, $K_2$ and $K_3$ given in \cref{sf} such that  \begin{align}
       \sqrt{2} H(\sigma^{-1}S_n,Z)&\lessapprox \frac{K_1}{D\sqrt{n}}+\frac{D}{2\sqrt{n}}+\frac{\log(n/D^2)}{4\sqrt{2}\sqrt{n}} (\tilde R-1)+\|S_{Y_1}(\sigma Y_1)\|_2\Big\{\frac{{K_2\tilde R}}{\sqrt{n}}+\frac{K_3}{\sqrt{n}}
\Big\}
   \\&=O\Big(\frac{\log(n)}{\sqrt{n}}\Big)
\end{align}
\end{theorem}

Note that an exact  and a tighter computable bound can be found in \cref{exact}. The latter is less interpretable but is useful for deriving as tight as possible quantiles bounds using \cref{eff_hellinger}. The proof can be found in \cref{hell_proof}.

\subsection{General bound for the Hellinger distance }
We remark that \cref{bound} establishes a rate of convergence for the Hellinger distance under moment conditions for the score function of $Y_1$. This result is novel. In this section we present the general bound we established and used to derive \cref{bound}. Our proof technique relies on the Ornstein-Ulbheck interpolation, the heat equation, and the Stein exchangeable pair method. The latter takes inspiration from the ideas \citet{bonis2015rates}  adapted to obtain a rate in the Wassertein distance. This provides us with an easily computable bound that only depends on the moments of the score function and the moments of the random variable itself. This is to contrast with the known bounds for the KL divergence (which upper-bounds the square of the Hellinger distance), which are either not in closed form \citep{bobkov2013rate}, require conditions on the spectral gap \citep{artstein2004rate} or depend on the moment of the Stein kernel \citep{ledoux2015stein} which is often significantly harder to upper-bound and does not always exists.

Let $S$ be a real-valued random variable with PDF $f_S\in \mathbb{C}^1(\mathbb{R})$ with support $\Omega_S\subset\mathbb{R}$. We write its score function $S_S(\cdot)$. Let $h_k(x)\defeq e^{x^2/2} \frac{\partial^k}{\partial x}e^{-x^2/2}$ designate the Hermite polynomials,  and denote $H_k\defeq h_k(Z)$ where $Z\sim \mathcal{N}(0,1)$ is a standard normal. We write the interpolated process as $$X^S_t\defeq e^{-t}S+\sqrt{1-e^{-2t}}Z.$$ We denote by $f^S_t$ the PDF of $X^S_t$ and define $S_{X^S_t}$ to be the score function of $X^S_t$. We remark that for all $t>0$ the function $f^S_t\in \mathbb{C}^{\infty}(\mathbb{R})$ is smooth and we write $I^S_t(\cdot)=\frac{f^{S''}_t(\cdot)}{f^S_t(\cdot)}$. For every $t\ge 0$ choose $S^t$ to be a random variable such that $(S,S^t)$ is an exchangeable pair. For all $t>0$ we define\begin{align}\label{tau_1}\tau^S_t\defeq\sum_{k=1}^{\infty}\frac{e^{-kt}}{k!\sqrt{1-e^{-2t}}^{k-1}}\mathbb{E}((S^t-S)^k|S)H_{k-1}(Z);\end{align}\begin{align}\label{tau_2}\tau^{',S}_t\defeq\sum_{k=1}^{\infty}\frac{e^{-kt}}{k!\sqrt{1-e^{-2t}}^k}\mathbb{E}((S^t-S)^k|S)H_k(Z).\end{align}
\begin{theorem}[Bounding Hellinger distance to a Gaussian]\label{basic_hell}
Assume that $S$ is standardized meaning that $\mathbb{E}(S)=0$ and that $\rm{var}(S)=1$. Suppose that $f_S\big|_{\delta\Omega_S}=0$. Then the following holds
\begin{align}
 \sqrt{2}  H(S,Z)
 &\le \int_0^{\infty}\Big\|I^S_t(X^S_t)-(X^S_t)^2+1-X^S_t(S_{X^S_t}-X^S_t)\Big\|_2dt
   \\&\le \inf_{\substack{D>0\\p>1}}\Big\{  \|S_S(S)\|_{\frac{2p}{p-1}}\int_0^{-\frac{1}{2}\log(1-\frac{D}{\sqrt{n}})} \Big\|\frac{Ze^{-2t}}{\sqrt{1-e^{-2t}}}-e^{-t}S+\tau_t^S\Big\|_{2p}dt
 \\&\qquad -\frac{1}{2}\log(1-\frac{D}{\sqrt{n}})+ \int_{-\frac{1}{2}\log(1-\frac{D}{\sqrt{n}})}^{\infty}\Big\|\frac{e^{-2t}}{{1-e^{-2t}}}(Z^2-1)-\frac{e^{-t}}{\sqrt{1-e^{-2t}}}SZ+\tau^{',S}_t\Big\|_2dt\Big\}
\end{align}
\end{theorem} This new upper-bound for the Hellinger distance is the key to the proof
of \cref{bound}. Outside of giving us a bound for $H(\tilde S_n,Z)$, this result can be used to obtain bounds on $H(S,Z)$ for any other random variable $S$. We remark that this upper bound will depend on the moments of $S$ and of its score function. To do so one can pick any specific $D>0$ and $p>1$ and compute the corresponding bound. Note that we use this idea to obtain \cref{bound} and that any arbitrary choice of $D$ and $p$ give us that $\sqrt{2}H(\tilde S_n,Z)$ decreases at a rate of $O\Big(\frac{\log(n)}{\sqrt{n}}\Big)$. However depending on the choices of those coefficients the constants in the bound of $H(\tilde S_n,Z)$ will change. We note that this result plays a similar role to bounds that had been developed for the KL-divergence or the Wassertein distance see \citep{barron1986entropy,otto2000generalization,ledoux2015stein}. The proof is presented in \cref{proof_basic_hell}.
\noindent In \citet{bonis2015rates}  similar ideas could be exploited to obtain a bound for the $p$-Wassertein distance $\wass (S,Z)$. Notably, it was established that  \begin{align}
\wass (S,Z)
   &\le \int_0^{\infty}\Big\|\frac{e^{-2t}}{\sqrt{1-e^{-2t}}}Z-{e^{-t}}S-\tau_t^S\Big\|_pdt.
\end{align} The Wassertein distance is a distance on the underlying geometry of the random variables. This is to contrast with the Hellinger distance which is
an information-theoretic distance that intimately depends on the PDF and its smoothness. This difference explains why the latter can be upper-bounded independently from the score function.
\subsection{Proof sketch for \cref{eff_hellinger}}
In each of the proof sketches to follow, we assume, without loss of generality, that $\sigma=1$.

The proof of \cref{eff_hellinger} is a consequence of the Cauchy-Swartz inequality. Indeed let $\varphi(\cdot)$ be the PDF of a standard normal random variable, and  $\tilde f_n$ the PDF of $S_n$. If we write $h(t)=\frac{\tilde f_n(t)}{\varphi(t)}$ then \begin{align}
    \P(S_n\ge u)-\Phi^c(u)&=\int_u^{\infty}(h(t)-1)\varphi(t)dt
    \\&\le\int_u^{\infty}(\sqrt{h(t)}-1)(\sqrt{h(t)}+1)\varphi(t)dt
\\&\overset{(a)}{\le}\int_u^{\infty}(\sqrt{h(t)}-1)^2h(t)\varphi(t)dt+\int_u^{\infty}(\sqrt{h(t)}-1)^2\varphi(t)dt
\end{align}
 where (a) is consequence of Cauchy-Swartz inequality. Using the definition of $H(\cdot,\cdot)$ and $\hat{Q}_n(\cdot)$ we obtain the desired result.

\subsection{Proof sketch for \cref{basic_hell}}
The proof of \cref{basic_hell} relies on the Ornstein-Ulbheck interpolation, the heat equation and the Stein exchangeable pair. To make this precise we define the interpolated process as $X^S_t\defeq e^{-t}S+\sqrt{1-e^{-2t}}Z$ where $Z$ is an independent standard normal random variable. As $t$ varies from $0$ to $\infty$ $X_t^S$ evolves from $S$ to $Z$. Indeed we remark that $X^S_0=S$ and $\lim_{t\rightarrow\infty} X^S_t\rightarrow Z$. For all $t>0$, as $X^S_t$ is the sum of $S$ with an independent normal it is a continuous random variable that admits a PDF $f_t^S$ that is smooth. We write $h_t^S(x)\defeq\frac{f_t^S(x)}{\varphi(x)}$.

\noindent Fix an arbitrary $T<\infty$. By definition we have
\begin{align}
    H(S,X^S_T)^2&=1-\int_{\mathbb{R}}\sqrt{f_S(x)}\sqrt{f^S_T(x)}dx.
\end{align}We remark that for all $x\in \mathbb{R}$ the function $t\rightarrow f_t^S(x)$ is differentiable. Therefore we have $\int_0^T\partial_t\sqrt{f^S_t(x)}dt=\sqrt{f^S_T(x)}-\sqrt{f_S(x)}$ which implies that\begin{align}\label{mika}
    H(S,X^S_{T})^2&=-\frac{1}{2}\int_{0}^{T}\int_{\mathbb{R}}\sqrt{f_S(x)}\partial_t\sqrt{f^S_t(x)}dxdt
\end{align}We use this characterization of the Hellinger distance to bound it in terms of information-theoretic quantities. Indeed, by the chain rule and the heat equation we obtain that \begin{align}\partial_t\sqrt{f^S_t(x)}&=\frac{1}{2\sqrt{f^S_t(x)}}\partial_t{f^S_t(x)}
=\frac{\varphi(x)}{2\sqrt{f^S_t(x)}}\partial_t\frac{f^S_t(x)}{\varphi(x)}
\\&=\frac{\varphi(x)}{2\sqrt{f^S_t(x)}}\partial_th_t^S(x)
=\frac{\varphi(x)}{2\sqrt{f^S_t(x)}}[{h^{S}_{t}}''(S)(x)-x{h^{S}_{t}}'(x)]
\\&=\frac{f_t^S(x)}{2\sqrt{f^S_t(x)}}\Big[\frac{{h^{S}_{t}}''(x)}{h_{t}^S(x)}-x\frac{{h^{S}_{t}}'(x)}{h_{t}^S(x)}\Big].
\end{align} Therefore by combining this with \cref{mika} we obtain that \begin{align}
    H(S,X^S_{T})^2&=-\frac{1}{2}\int_{0}^{T}\mathbb{E}\Big(\frac{\sqrt{f_S(X_t^S)}}{2\sqrt{f^S_t(X_t^S)}}\Big[\frac{{h^{S}_{t}}''(X_t^S)}{h_{t}^S(X_t^S)}-X_t^S\frac{{h^{S}_{t}}'(X_t^S)}{h_{t}^S(X_t^S)}\Big]\Big)dt.
\end{align}Using the fact that  $\mathbb{E}\Big(\frac{{h^{S}_{t}}''(X^S_t)}{h_t^S(X^S_t)}-X^S_t\frac{{h^{S}_{t}}'(X^S_t)}{h_t^S(X^S_t)}\Big)=0$, the Cauchy-Swartz inequality and the definition of the Hellinger distance we then establish  that
\begin{align}H(S,X_T^S)^2&
    \le \frac{1}{\sqrt{2}}  \sup_{t\le T}H(S,X_t^S)\int_0^{T}\Big\|\frac{{h^{S}_{t}}''(X_t^S)}{h_{t}^S(X_t^S)}-X_t^S\frac{{h^{S}_{t}}'(X_t^S)}{h_{t}^S(X_t^S)}\Big\|_2dt.
\end{align}As this holds for any arbitrary $T<\infty$ this implies that
\begin{align}H(S,Z)&
    \le \frac{1}{\sqrt{2}}\int_0^{\infty}\Big\|\frac{{h^{S}_{t}}''(X_t^S)}{h_{t}^S(X_t^S)}-X_t^S\frac{{h^{S}_{t}}'(X_t^S)}{h_{t}^S(X_t^S)}\Big\|_2dt.
\end{align}This bound  can be seen as the equivalent of the De Bruijn identity  \citep[see, e.g.,][]{barron1986entropy} for the KL-divergence or similar expressions  obtained for $p$-Wassertein distance in the celebrated work of \citet{otto2000generalization} and  \citet{ledoux2015stein}.
As we don't know $h_t^S$, the right hand side of this equation is not easy to directly control. To go around this issue we re-express $\frac{{h^{S}_{t}}''(x)}{h_{t}^S(x)}-x\frac{{h^{S}_{t}}'(x)}{h_{t}^S(x)}$ in terms of the score function $S^S_{X_t^S}$ and $I_t^S$, which we can do as a simple consequence of the chain-rule. The advantage of this is that using the Stein identity we can re-express the score function and $I_t^S$. Indeed we remark that $S^S_t(X^S_t)$ is the unique $X^S_t$-measurable random variable such that for all $\phi\in \mathbb{C}^{\infty}(\mathbb{R})$ we have\begin{align}
    \mathbb{E}(S^S_t(X^S_t)\phi(X^S_t))=\mathbb{E}(\phi'(X^S_t)).
\end{align}Similarly we remark that $I^S_t(X^S_t)$ is the unique $X^S_t$-measurable random variable such that for all $\phi\in \mathbb{C}^{\infty}(\mathbb{R})$ we have\begin{align}
    \mathbb{E}(I^S_t(X^S_t)\phi(X^S_t))=\mathbb{E}(\phi''(X^S_t)).
\end{align}Therefore using the Stein identity we can prove that
\begin{align}&
    S_{X_t^S}^S(X_t^S)=\begin{cases}
        e^t\mathbb{E}(S_S(S)|X_t^S)\\\frac{1}{\sqrt{1-e^{-2t}}}\mathbb{E}(Z|X_t^S)
    \end{cases}
    \\&I_t^S(X_t^S)=\begin{cases}
        \frac{e^t}{\sqrt{1-e^{-2t}}}\mathbb{E}(S_S(S)Z|X_t^S)\\\frac{1}{{1-e^{-2t}}}\mathbb{E}(Z^2-1|X_t^S).
    \end{cases}
\end{align}
The last step of the proof is to notice that $\tau^{',S}_t$ and $\tau^S_t$ are centered with respect to the conditional expectation: $\mathbb{E}(\tau_t^S|X_t^S)=\mathbb{E}(\tau_t^{',S}|X_t^S)=0$. Indeed we establish that for all smooth function $\Phi\in \mathbb{C}^{\infty}(\mathbb{R})$ we have
\begin{align}
    \mathbb{E}\big(\tau^{',S}_t\Phi(X_t^S)\big)&=\mathbb{E}\Big(\Phi(X_t^S)\Big)-\mathbb{E}\Big(\Phi(e^{-t}S^t+\sqrt{1-e^{-2t}}Z)\Big)\\&\overset{(a)}{=}0.
\end{align} where to obtain (a) we use the fact that $(S,S^t)$ is an exchangeable pair. As this holds for arbitrary $\Phi$ this directly implies that $\mathbb{E}(\tau^{',S}_t|X_t^S)=0$. Similarly we prove that $\mathbb{E}(\tau^S_t|X_t^S)=0$.
This establishes \cref{basic_hell}.

\subsection{Proof sketch for \cref{bound}}
To prove \cref{bound} we build an exchangeable pair $(S_n,S_n^t)$ in the following way: We first start by drawing an independent copy $(Y'_i)$ of $(Y_i)$. We set a $\kappa>0$ and write $c(t)=\sqrt{\kappa}\sqrt{1-e^{-2t}}$. Then for every $t>0$ we choose an index $I\sim\rm{unif}\{1,\dots,n\}$ at random and change the random variable $Y_i$ by its independent copy $Y'_i$ if and only if both $|Y_i/\sqrt{n}|\le c(t)$ and $|Y'_i/\sqrt{n}|\le c(t)$:
$$S_n^t\defeq S_n+\frac{1}{\sqrt{n}}\mathbb{I}(|Y_i/\sqrt{n}|,|Y_i/\sqrt{n}|\le c(t)) \big(Y'_i-Y_i\big).$$ As the observations $(Y_i)$ are i.i.d we remark that $(S_n,S_n^t)$ is indeed an exchangeable pair. The advantage of this choice of exchangeable pair is that we have $$\mathbb{E}(S_n^t|S_n)\approx S_n,\qquad \frac{1}{2}\mathbb{E}((S_n^t-S_n)^2|S_n)\approx 1.$$This will allow us to obtain small upper bounds for $\|\frac{e^{-2t}}{\sqrt{1-e^{-2t}}}Z-{e^{-t}}\tilde S_n+\tau_t\big\|_{2p''}$ and $\Big\|\frac{e^{-2t}}{{1-e^{-2t}}}(Z^2-1)-\frac{e^{-t}}{\sqrt{1-e^{-2t}}}\tilde S_nZ+\tau^{'}_t\Big\|_2$ and thereore control $H(S_n,Z)$.

\section{Efficient Concentration with Stein Kernels}\label{stein_sec}In the previous section, we showed how we can use the Hellinger distance to obtain an efficient concentration inequality when \cref{score_function_existance} holds. In this section, we will see how one can also obtain efficient concentration inequalities under the alternative condition than the random variables $(Y_i)$ admit a Stein kernel. This can be useful notably for a certain number of discrete random variables that have a Stein kernel but not a score function. However, we note that not all random variables that satisfy \cref{score_function_existance} will also admit a Stein kernel or have it nicely controlled. Therefore the two approaches are complementary. Finally, we will see in \cref{stein_thm_3} how when both exist we can exploit this to obtain an even tighter concentration inequality.

The notion of a Stein kernel was first used in \citet{cacoullos1992lower}, although the term ``{Stein kernel}'' was introduced by \citet{ley2017stein}. A function $\tau$ is called the Stein kernel of a standardized random variable $X$ if for all functions $\phi\in \mathbb{C}^1(\mathbb{R})$ we have:
\begin{align}\mathbb{E}(\phi(X)X)=\mathbb{E}(\tau(X)\phi'(X)).\end{align} If $X$ is a continuous random variable with PDF $f_X$, and if $f_X$ has as support a (potentially infinite) connected interval $(a,b)$ then, under some regular stability conditions on $f_X$, the Stein kernel is given by $\tau(x)=\frac{1}{f_X(x)}\int_x^{\infty}tf_X(t)dt$ \citep{ley2017stein}. Notably, if $X$ is a standard normal random variable then its Stein kernel is given by $\tau(X)=1$. When the random variable $X$ does not possess such a density, the Stein kernel sometimes still exists, this is for example the case if the random variables satisfy a Poincare inequality \citep{fathi2019stein}. In other cases, the existence has been established on a case-by-case basis.

This is notably true if $X$ is a symmetric random variable with support $\{-1,+1\}$ \citep{chatterjee2012new}. See \citet{ley2017stein} for other examples of Stein kernels and discussion about its existence. However, it is important to note that, in general, a Stein kernel does not always exist (even when $X$ is continuous), and general conditions for its existence are not well determined.

The Stein kernel can measure the distance between $X$ and a standard normal random variable by defining the \emph{Stein discrepancy}:
$$\mathcal{S}(X||Z)\defeq\|\tau(X)-1\|_2.$$
Similarly for $p\ge 2$, we can define the following distance $\mathcal{S}_p(X||Z)\defeq\|\tau(X)-1\|_p$.
 While the Stein kernel can be hard to control, it has proven to be a useful quantity. Notably, \citet{ledoux2015stein} has explored how it can be used to bound the $p$-Wassertein distance and the KL divergence between a random variable and a standard normal random variable. It has also been used in \citet{ledoux2015stein} to upper-bound the moments of Lipchitz functions of $S_n$. In this section, we established how it can also be used to obtain efficient concentration inequalities.

\def$(\tau_{\tilde Y_1},p')${$(\tau_{\tilde Y_1},p')$}
\addtocounter{assumption}{-1}
\begin{assumption}
    \label{stein_moment} Assume that $\sigma^{-1}Y_1$ admit a Stein kernel $\tau_{\tilde Y_1}$ and that $\|\tau_{\tilde Y_1}(\sigma^{-1}Y_1)\|_{p'}<\infty$.
\end{assumption}In the following we are going to take $\hat{Q}_n(x)$ to be such that for all $x>0$$$\sup_{t\ge 0}\P(e^{-t}\tilde S_n+\sqrt{1-e^{-2t}}Z\ge x)\le \hat{Q}_n(x).$$
We prove in \cref{quantile_t} that $\hat{Q}_n(x)$ can be chosen to be the Hoeffding bound for $R^2$ sub-Gaussian random variables or the upper bound obtained in \cref{cpt_concentration}. For all $p>1$ we write $$D_p(\tau_{\tilde Y_1})\defeq\min \begin{cases}
 A_p\|\tau_{\tilde Y_1}(\sigma^{-1}Y_1)-1\|_2+A_{n,p}^*\|\tau_{\tilde Y_1}(\sigma^{-1}Y_1)-1\|_p
 \\ \sqrt{p-1}\|\tau_{\tilde Y_1}(\sigma^{-1}Y_1)-1\|_p
 \end{cases}.$$
\begin{theorem}[Efficient Stein kernel tail bound]\label{stein_thm_1}Let $p'\ge 2$. Suppose that the random variables $(Y_i)$ satisfy \cref{assump:bounded-dev} and \cref{stein_moment}. %
 Choose $\alpha>0$ and let $p\ge 1$ and $q\ge 2$. Define $u_{\alpha}(x)\defeq\frac{1}{\alpha^2}\int^{x+\alpha}_x\int^{t+\alpha}_{t}\mathbb{I}(z\ge u)dz dt$ and let $Z\sim N(0,1)$ then we have
\begin{align}&\label{nulle_cab}
     \P(S_n\ge \sigma u)\\&\le\mathbb{E}(u_{\alpha}(Z))+\frac{D_p(\tau_{\tilde Y_1})}{\sqrt{n}}\min\begin{cases}{\|Z\|_p}\Big[\Big\|u_{\alpha}'(Z)\Big\|_{\frac{p}{p-1}}+\frac{\pi \mathcal{W}_{\frac{pq}{(p-1)}}(G,\tilde S_n) \hat{Q}_n(u-2\alpha)^{\frac{(p-1)(q-1)}{pq}}}{4\alpha^2}\Big]
     \\\frac{1}{2\alpha^2}\hat{Q}_n(u-2\alpha)^{\frac{(p-1)}{p}}
     \end{cases}\\&<\P(Z\ge u-2\alpha)+\frac{D_p(\tau_{\tilde Y_1})}{\sqrt{n}}\min\begin{cases}\|Z\|_p\Big[\frac{1}{\alpha}\P(Z\in [u-2\alpha,u])^{\frac{p-1}{p}}+\frac{\pi}{4\alpha^2}\mathcal{W}_{\frac{pq}{(p-1)}}(G,\tilde S_n) \\\qquad\quad\hat{Q}_n(u-2\alpha)^{\frac{(p-1)(q-1)}{pq}}\Big]\\\frac{1}{2\alpha^2}\hat{Q}_n(u-2\alpha)^{\frac{(p-1)}{p}}
     \end{cases}
 \end{align}
 \begin{rem}
     We note that a tighter but less interpretable bound is presented in \cref{tighter_stein}. Moreover we remark that as $p$ increases the terms $\Big\|u_{\alpha}'(Z)\Big\|_{\frac{p}{p-1}}$ and $\hat{Q}_n(u-2\alpha)^{\frac{(p-1)(q-1)}{pq}}$ decrease, and that both are asymptotically bounded by respectively $\P(Z\in [u-2\alpha, u])\le 2\alpha \varphi(u-2\alpha)$ and $e^{-\frac{(u-2\alpha)^2}{2R^2}}$.
 \end{rem}
We remark that similarly to \cref{basic_hell} we can obtain a similar efficient  two-sided tail bound. This is presented in \cref{double_stein}. The proof can be found in \cref{proof_stein_thm1}. See \cref{stein_sketch} for a proof sketch.

\end{theorem}

{Let us compare this result to what has been previously obtained in the literature.} Under the condition that the Stein kernel $\tau_{\tilde Y_1}(\cdot)-1$ is almost surely bounded by $\mathcal{S}_{\infty}(\sigma^{-1}Y_1||Z)$ then \citet{chatterjee2012new} proved that there is $G\sim N(0,1)$ normally distributed such that for all $\theta>0$$$\mathbb{E}\big(e^{\theta |\tilde S_n-G|}\big)\le 2 e^{2\theta^2\mathcal{S}_{\infty}(\sigma^{-1}Y_1||Z)^2}.$$ $G$ is called a strong embedding of $\tilde S_n$. If the Stein kernel is not bounded or does not exist \citet{bhattacharjee2016strong} but $\mathbb{E}(Y_1^3)=0$, then \citet{bhattacharjee2016strong} proposed to study a slightly smoothed empirical average and obtained a similar inequality. Using a union bound and a Markov inequality, this can be used to obtain efficient concentration inequalities for $S_n$:
\begin{theorem}[Alternative Stein kernel tail bound]
    \label{thm_stein_2} Suppose that the random variables satisfy \cref{assump:bounded-dev} and \cref{stein_moment} with $p'=\infty$.
    Then for all $u>0$ and $0<\alpha<u/2$ the following holds \begin{align}\label{bd_st_2}\P(S_n\ge \sigma u)\le \Phi^c(u-2\alpha)+2e^{-\frac{n\alpha^2}{2\mathcal{S}_{\infty}(\sigma^{-1}Y_1||Z)^2}}.\end{align}\end{theorem}

    We remark that \cref{thm_stein_2} require stronger moment conditions on the Stein kernel than \cref{stein_thm_1} does. Moreover for illustration if we choose $p=q=2$ (which is not necessarily the optimal choice) then \cref{stein_thm_1} tells us that\begin{align}&\label{bonne_dec}
     \P(S_n\ge \sigma u)\\&\le \mathbb{E}(u_{\alpha}(Z))+\frac{1}{\sqrt{n}}\|\tau_{\tilde Y_1}(\sigma^{-1}Y_1)-1\|_2
\Big[\Big\|u_{\alpha}'(Z)\Big\|_{2}+\frac{\pi \mathcal{W}_4(G,\tilde S_n)}{4\alpha^2} \hat{Q}_n(u-2\alpha)^{\frac{1}{4}}\Big]
 \end{align} To contrast this with the bound obtained in \cref{thm_stein_2}, we note that $\mathbb{E}(u_{\alpha}(Z))\le \Phi^c(u-2\alpha)$. Moreover we also remark that $$\Big\|u_{\alpha}'(Z)\Big\|_{2}\le \alpha^{-1}\sqrt{\P(Z\in [u-2\alpha, u])}.$$ Therefore  we remark that the right-hand side of \cref{bonne_dec} is Sub-Gaussian in $u$.
 In contrast we notice that the second term in \cref{bd_st_2},  $e^{-\frac{n\alpha^2}{2\mathcal{S}_{\infty}(\sigma^{-1}Y_1||Z)^2}}$, does not depend on $u$.
 Therefore for large $u$ and moderate size $n$ the bound in \cref{stein_thm_1} will tend to be tighter.  Finally using \cref{lemm:dino_cpt} we know that there is a constant $K$ such that $\mathcal{W}_4(G,\tilde S_n)\le \frac{\tilde R K}{\sqrt{n}}[1+o_n(1)]$.

We observe that if \cref{score_function_existance} also holds then by using \citet{ledoux2015stein} we can exploit \cref{eff_hellinger} to obtain an efficient  concentration inequality that is exactly sub-Gaussian in $u$.
 \begin{theorem}[Efficient Hellinger-Stein kernel tail bound]\label{stein_thm_3} Set $p'=2$.
     Suppose that the random variables $(Y_i)$ satisfy \cref{assump:bounded-dev}, \cref{score_function_existance}, \cref{moment_score} and \cref{stein_moment}. Let $\hat{Q}_n(u)$ and $\hat{Q}^d_n(u)$ be an any arbitrary upper bound to respectively $\P(S_n\ge \sigma u)\le \hat{Q}_n(u)$ and $\P(|S_n|\ge \sigma u)\le \hat{Q}^d_n(u)$. Suppose that $\|S_{Y_1}(\sigma Y_1)\|_2<\infty$ and that $\mathcal{S}(\sigma^{-1}Y_1||Z)<\infty$. Then for all $u\ge 0$ the following bounds hold
     \begin{align}
         \P(S_n \ge \sigma u)\le \Phi^c(u)+&\frac{\sqrt{2}\mathcal{S}_2(\sigma^{-1}Y_1||Z)}{\sqrt{n}}\sqrt{\log\Big(1+
        \frac{n\|S_{Y_1}(\sigma Y_1)-\sigma^{-1}Y_1\|_2^2}{\mathcal{S}_2(\sigma^{-1}Y_1||Z)^2}\Big)}\\&\times\Big[\sqrt{\hat Q_n(u)}+\sqrt{\Phi^c(u)}\Big]
     \end{align} and
     \begin{align}
         \P(|S_n| \ge \sigma u)\le 2\Phi^c(u)+&\frac{\sqrt{2}\mathcal{S}_2(\sigma^{-1}Y_1||Z)}{\sqrt{n}}\sqrt{\log\Big(1+
        \frac{n\|S_{Y_1}(\sigma Y_1)-\sigma^{-1}Y_1\|_2^2}{\mathcal{S}_2(\sigma^{-1}Y_1||Z)^2}\Big)}\\&\times\Big[\sqrt{\hat Q^d_n(u)}+\sqrt{2\Phi^c(u)}\Big]
     \end{align}
 \end{theorem}

 See \cref{stein_sketch2} for a proof sketch and \cref{proof_stein_3} for the proof. We note that the rate obtained in \cref{stein_thm_3} is sub-Gaussian in $u$ and that the reminder terms decreases at a rate of $\sqrt{\log(n)/n}$. Therefore if in addition to the Stein Kernel we can also control the score function then we can obtain a tighter rate than in \cref{thm_stein_2} and \cref{stein_thm_1}.
\subsection{Proof sketch for \cref{stein_thm_1}}\label{stein_sketch}We note that $u_{\alpha}(x)\le \mathbb{I}(x\le u)$. By the monotony of the expectation this directly implies that $\mathbb{E}(u_{\alpha}(\tilde S_n))\le \P(\tilde S_n\ge u)$. Therefore if we obtain a bound on $\big|\mathbb{E}(u_{\alpha}(\tilde S_n))-\mathbb{E}(u_{\alpha}(Z))\big|$, we can use it to obtain a bound on $\P(\tilde S_n\ge u)$.
The rest of the proof of \cref{stein_thm_1} consists of using the Ornstein-Ulbheck interpolation and the heat equation.

To see this, we denote by $\tau_{\tilde S_n}(\cdot)$ the Stein kernel of $\tilde S_n$ and take $Z\sim N(0,1)$ to be an independent standard normal random variable. We define $P_tu_{\alpha}(x)\defeq\mathbb{E}_{Z}(u_{\alpha}(e^{-t}x+\sqrt{1-e^{-2t}}Z))$. We remark that $t\rightarrow P_tu_{\alpha}$ interpolates between $P_0u_{\alpha}(x)=u_{\alpha}(x)$ and $P_{\infty}u_{\alpha}(x)=\mathbb{E}(u_{\alpha}(Z)).$
 Therefore we have:
 \begin{align}\Big|\mathbb{E}(u_{\alpha}(\tilde S_n))-\mathbb{E}(u_{\alpha}(Z))\Big|&=\Big|\int_0^{\infty} \partial_t\mathbb{E}(P_tu_{\alpha}(\tilde S_n))dt\Big|
 \\&\overset{(a)}{\le} \Big|\int_0^{\infty} \mathbb{E}(\partial_t^2 P_tu_{\alpha}(\tilde S_n)-\tilde S_n\partial_t P_tu_{\alpha}(\tilde S_n))dt\Big|
 \\&\overset{(b)}{\le}  \Big|\int_0^{\infty} \mathbb{E}(\partial_t^2 P_tu_{\alpha}(\tilde S_n)(1-T_{\tilde S_n}(\tilde S_n)))dt\Big|
 \end{align} where to obtain (a) we used the heat equation satisfied by the Ornstein-Uhlenbeck process and where (b) is a consequence of the definition of Stein kernels.

It is well known (see e.g \citep{ledoux2015stein}) that $\tau_{\tilde S_n}(\tilde S_n)=\mathbb{E}\big(\frac{1}{n}\sum_{i=1}^n\tau_{\tilde Y_1}(Y_i)|\tilde S_n\big)$. Therefore as $\mathbb{E}(\tau_{\tilde Y_1}(Y_1))=1$ for all $p\ge 1$ there is a constant $K_p$ such that $$\|\tau_{\tilde S_n}(\tilde S_n)-1\|_p\le \|\frac{1}{n}\sum_{i=1}^n\tau_{\tilde Y_1}(Y_i)-1\|_p\le \frac{K_p}{\sqrt{n}}\|\tau_{\tilde Y_1}(Y_1)-1\|_p.$$
Therefore if we can control the size of $\partial_t^2 P_t u_{\alpha}(\tilde S_n)$ then we will obtain that $\big|\mathbb{E}(u_{\alpha}(\tilde S_n))-\mathbb{E}(u_{\alpha}(Z))\big|=O(\frac{1}{\sqrt{n}})$. The remainder of the proof consist in controlling $\partial_t^2 P_t u_{\alpha}(\tilde S_n)$. This is what we do thanks to the Stein identity and the Wassertein distance.
\subsection{Proof sketch of \cref{stein_thm_3}}\label{stein_sketch2}

The proof of \cref{stein_thm_3} is based on \cref{eff_hellinger} which guarantees that \begin{align}
 \P(S_n\ge \sigma u)\le\Phi^c(u)+ \sqrt{2}H(\sigma ^{-1}S_n,Z)\Big[\sqrt{\hat Q_n(u)}+\sqrt{\Phi^c(u)}\Big]
\end{align} and Theorem 2.2 of \citet{ledoux2015stein} which upper bounds the KL-divergence $KL(\tilde S_n||Z)$ in terms of the Stein kernel.
\section{Efficient Concentration with Wasserstein Approximation}\label{simple}
In the two previous sections, we showed how to obtain efficient concentration given auxiliary knowledge of a score function or Stein kernel.
In this section, we establish efficient concentration without any additional knowledge beyond \cref{assump:bounded-dev}.
To do so, we measure how far $S_n$ is from a normal by the $p$-Wassertein distance $\wass (\cdot,\cdot)$, defined in the following way: $$\wass (\nu,\mu)= \inf_{\substack{\gamma\in \varphi(\nu, \mu)}}\mathbb{E}_{(X,Y)\sim \gamma}\big(|X-Y|^p\big)^{1/p},$$
where $\gamma(\nu, \mu)$ is the set of all possible couplings of the probability distributions $\nu$ and $\mu$ on $\reals$.

\subsection{Efficient tail bounds}

When \cref{assump:bounded-dev} holds we obtain in \cref{efficient_tail} below that, for each $u > 0$,
\begin{talign}&
\P(S_n\ge \sigma u)
\le \Phi^c\big(u-\frac{5.5R\log(n)}{\sigma \sqrt{n}}\big)[1+o_n(1)]+\frac{5.5R}{\sqrt{n}\sigma}\varphi\big(u-\frac{5.5R\log(n)}{\sigma\sqrt{n}}\big)[1+o_n(1)].
\end{talign}
Notably, this bound is asymptotically exact, converging to the correct asymptotic tail bound  $\Phi^c(u)$ as $n\to\infty$, with an error term that is asymptotically sub-Gaussian in $u$.
To state our result more precisely, we define the growth constant
\begin{talign}
     K&\defeq \frac{1}{\sqrt{0.4}}\Big[R+\frac{\sqrt{0.4e}\sigma}{2\sqrt{2}}\Big]+\frac{2\sqrt{e}R}{\sqrt{1.2}}\Big[e^{{0.8}}-1.8\Big]\Big[2-\frac{2\sigma}{3R}\Big]
\approx 3.71R,
 \end{talign}
the interpolation parameter
\newcommand{\rhoopt}[1][u]{\rho^\star_{n,#1}}

\begin{talign}
\rhoopt &\defeq\frac{\sigma}{eKu}\Big(\sqrt{n+\frac{2eK}{\sigma^2}\big(\sqrt{n}u\sigma-eK(2+\log(\frac{\sqrt{2\pi n\sigma^2}}{Ke}))\big)}-\sqrt{n}\Big)
\\
&\approx 1-\frac{eK\sigma}{\sqrt{n}u}\log\Big(\frac{\sqrt{n}\sigma\sqrt{2\pi}}{eK}\Big)
\approx 1-\frac{5.05R\log(0.25n\sigma^2/R^2)\sigma}{\sqrt{n}u},
 \end{talign}
and the correction power
\begin{talign}
&L_{n,u}\defeq \frac{(1-\rhoopt )\sqrt{n}u\sigma}{Ke}-2\approx 5.05R\log(0.25n\sigma^2/R^2).
\end{talign}
To first order, \cref{efficient_tail} establishes that $\P(S_n\ge \sigma u)\le \Phi^c\big({\rhoopt u}\big)[1+o_n(1)]+\frac{eK}{\sqrt{n}\sigma}\varphi({\rhoopt u})[1+o_n(1)].$
To make this bound exact we incorporate a small correction term $r_{n,u}$ of order  $\frac{R^2\log(n)^{5/2}}{\sigma^2 n}\varphi({\rhoopt u})$ defined in  \cref{grasse1}.
Under this notation, we have the following efficient tail bound for $S_n$, proved in \cref{suisse}.
\begin{theorem}[Efficient tail bound]\label{efficient_tail}
If $S_n$ satisfies \cref{assump:bounded-dev} and
$2 \leq L_{n,u} \leq 1+\frac{0.4\sigma^2 n}{R^2}$, then
\begin{talign}
\P(S_n\ge \sigma u)&\ \ \le\ \ \Phi^c({\rhoopt u})+\frac{eK}{\sqrt{n}\sigma}\varphi({\rhoopt u})\big[1+  r_{n,u}\big]^{L_{n,u}}
\\
&\underset{n\rightarrow\infty}{\approx}\Phi^c\big(u-\frac{5.5R\log(n)}{\sigma\sqrt{n}}\big)+\frac{5.5R}{\sqrt{n}\sigma}\varphi\big(u-\frac{5.5R\log(n)}{\sigma\sqrt{n}}\big)+O(\frac{R^2\log(n)^{5/2}}{\sigma^2 n}\varphi({\rhoopt u})).
\end{talign}
\end{theorem}

\begin{rem}
The constant $5.5$ can be significantly reduced at the price of a more complicated expression.
We also present a tighter but less interpretable bound that holds for all $u \geq 0$ in \cref{tighter}.
\end{rem}

\subsection{Efficient quantile bounds}

Similarly to how the Wassertein distance can be used to obtain efficient tail bounds it can also be used to obtain efficient quantile bounds. To do so we once again tightly bound the distance between $S_n$ and its Gaussian limit. The quantile bounds we obtain in \cref{efficient_quantile} are of the order \begin{align}  q_n(R,\delta,\sigma)&\underset{n\rightarrow \infty}{\sim}-\sigma \Phi^{-1}\Big({\delta-\frac{eK}{\sigma\sqrt{n}}\varphi(\Phi^{-1}(\delta))}\Big)+ \frac{eK }{\sqrt{n}} \log\big(\frac{\sqrt{n}\sigma}{eK}\varphi(\Phi^{-1}(\delta))^{-1}\big)
    \\&\approx-\sigma \Phi^{-1}\Big({\delta-\frac{10.1R}{\sigma\sqrt{n}}\varphi(\Phi^{-1}(\delta))}\Big)+ \frac{10.1R }{\sqrt{n}} \log\big(\frac{\sqrt{n}\sigma}{10.1R}\varphi(\Phi^{-1}(\delta))^{-1}\big).
 \end{align}
 Notably, as the sample size increases these bounds converge to the true asymptotic quantiles $-\sigma \Phi^{-1}(\delta)$, guaranteeing efficiency.
 To state our result in full, we introduce
 the shorthand
$$A^n_{\delta}\defeq \log\Big(\frac{\sqrt{n}\sigma}{eK}\varphi(\Phi^{-1}(\delta))^{-1}\Big)$$
and an explicit correction term $s_{\delta,n}$ of order $\frac{R^2\log(n)^{5/2}}{n\sigma}$ defined in \cref{grasse2}.
With this notation in place, we have the following efficient quantile bound for $S_n$, proved in \cref{suisse}.
\begin{theorem}[Efficient quantile bound]\label{efficient_quantile}
If $S_n$ satisfies \cref{assump:bounded-dev}, $A_{\delta}^n\ge 2$, and $0\le\frac{ R^2(A_{\delta}^n-1)}{0.4n\sigma^2}\le 1$ then
$P\big(S_n\ge q_n(R,\delta,\sigma)\big)\le \delta$
for
 \begin{align}
q_n(R,\delta,\sigma&)\defeq \sigma \Phi^{-1}\Big({\delta-\frac{eK}{\sigma\sqrt{n}}\varphi(\Phi^{-1}(\delta))}\Big)+ \frac{e\big[A^n_
    {\delta}+2\big]}{\sqrt{n}}\Big[K+s_{\delta,n}\Big]
    \\&\underset{n\rightarrow \infty}{\approx}-\sigma \Phi^{-1}\Big({\delta-\frac{10.1R}{\sqrt{n}}\varphi(\Phi^{-1}(\delta))}\Big)+ \frac{10.1 R }{\sqrt{n}} \log\big(\frac{\sqrt{n}\sigma}{10.1R}\varphi(\Phi^{-1}(\delta))^{-1}\big).
 \end{align}
\end{theorem}
\begin{rem}
This bound can be used to build efficient confidence intervals with nominal coverage. Moreover, the constant 10.1 can be significantly reduced at the price of a more complicated expression.
In \cref{tighter}, we present a tighter but less interpretable quantile bound (\cref{cpt-quantile}) that holds for all $\delta \in (0,1)$.
\end{rem}

\subsection{Tighter computable bounds}
\label{tighter}

In this section we propose tighter concentration inequalities and quantile approximations for $S_n$ than the ones presented in
 \cref{simple}.
 While these bounds are less interpretable, they are straightforwardly computed and hence can be used to produce tighter tail bounds and confidence intervals in practice.
First we obtain a tighter bound for $\P(S_n\ge  \sigma u)$, proved in  \cref{last_night}.
\begin{theorem}[Tighter efficient tail bound] \label{cpt_concentration}
If $S_n$ satisfies \cref{assump:bounded-dev}, then, for any auxiliary upper bounds $\onetail(u)$ and $\twotail(u)$ on $P\big(S_n\ge \sigma u\big)$ and $P\big(|S_n|\ge \sigma u\big)$ respectively, we have
\begin{align}
    P\big(S_n\ge \sigma u\big)&\le\min\Big(\inf_{\substack{p\in [2,\infty)\bigcap\{1\},\, \rho\in (0,1)}} \Phi^c({\rho u}{}) +\frac{\omega_p^{R}(\sigma)^p}{ (1-\rho)^p(u \sigma )^p},\  \onetail(u)\Big) \qtext{and} \\
    P\big(|S_n|\ge \sigma u\big)&\le\min\Big(\inf_{\substack{p\in [2,\infty)\bigcap\{1\},\, \rho\in (0,1)}} 2\Phi^c({\rho u}{}) +\frac{\omega_p^{R}(\sigma)^p}{ (1-\rho)^p(u \sigma )^p},\  \twotail(u)\Big)
\end{align}
for $\omega_p^{R}(\sigma)$ a computable bound on $\frac{1}{\sigma}\wass (S_n, \mathcal{N}(0,\sigma^2))$ defined in  \cref{cabris_sec} that is non-decreasing in $\sigma$.
\end{theorem}
\begin{remark}[Auxiliary tail bounds]
We include the auxiliary bounds  $\onetail(u)$ and $\twotail(u)$ to emphasize that our efficient  bounds can be paired with any valid tail bounds to simultaneously reap the large-sample benefits of the former and the small-sample benefits of the latter.
The Bernstein bounds $\onetail(u) = e^{-\frac{u^2}{2(1+Ru/3\sigma)}}$
and
$\twotail(u) = 2 e^{-\frac{u^2}{2(1+Ru/3\sigma)}}$
provide a convenient default choice.
\end{remark}
By choosing $p=L_{n,u}$ and  $\rho=\rhoopt[u]$ in \cref{cpt_concentration} we obtain a tightened version of the inequality in \cref{efficient_tail}.  Further improvements are obtained in practice by explicitly optimizing over $p$ and $\rho$.
Next we obtain tighter bounds for the quantiles of $S_n$, proved in \cref{last_night}.

\begin{theorem}[Tighter efficient quantile bound]\label{cpt-quantile} If $S_n$ satisfies \cref{assump:bounded-dev},
and some auxiliary deterministic bounds
$\oneq$ and $\twoq$ satisfy $P\big(S_n\ge \oneq(R,\delta,\sigma)\big) \leq \delta$ and $P\big(|S_n|\ge \twoq(R,\delta,\sigma)\big) \leq \delta$,
then
$$\P(S_n\ge q_n(R,\delta,\sigma))\le \delta,\qquad \P(|S_n|\ge q^d_n(R,\delta,\sigma))\le \delta$$
for $\omega_p^{R}(\sigma)$ a computable bound on $\frac{1}{\sigma}\wass (S_n, \mathcal{N}(0,\sigma^2))$ defined in \cref{cabris_sec}  that is non-decreasing in $\sigma$ and
\begin{align}\label{eq:tighter_efficient_quantile}&
q_n(R,\delta,\sigma)\defeq \min\Big(\inf_{\substack{p\ge 2,\, \rho\in (0,1)}}\frac{\sigma \omega_p^{R}(\sigma)}{(\delta(1-\rho))^{1/p}}-\sigma \Phi^{-1}(\delta \rho), ~\oneq(R,\delta,\sigma) \Big),
\\&q^d_n(R,\delta,\sigma)\defeq \min\Big(\inf_{\substack{p\ge 2,\, \rho\in (0,1)}}\frac{\sigma \omega_p^{R}(\sigma)}{(\delta(1-\rho))^{1/p}}-\sigma \Phi^{-1}(\frac{\delta \rho}{2}), ~\twoq(R,\delta,\sigma) \Big).
\end{align}
\end{theorem}
\begin{remark}[Auxiliary quantile bounds]\label{aux_q_bounds}
We again highlight that our efficient  bounds can be paired with any valid quantile bounds $\oneq(R,\delta,\sigma)$ and $\twoq(R,\delta,\sigma)$ to inherit the benefits of each.
Convenient defaults are provided by the minimum of the Bernstein quantiles \citep[Thm.~2.10]{boucheron2013concentration} and the Berry-Esseen quantiles \citep{shevtsova2011absolute},
\begin{talign}
\oneq(R,\delta,\sigma)
    &=
    \min\!\big(\frac{R}{3\sqrt{n}} \log(1/\delta)  + \sigma\sqrt{2 \log(1/\delta)},
    -\sigma \Phi^{-1}(\delta - \frac{\min(.3328 ( \tilde R + .429 ), .33554 ( \tilde R + .415 ))}{\sqrt{n}})
    \big), \\
\twoq(R,\delta,\sigma)
    &=
    \min\!\big(\frac{R}{3\sqrt{n}} \log(2/\delta)  + \sigma\sqrt{2 \log(2/\delta)},
    -\sigma \Phi^{-1}(\frac{\delta}{2} - \frac{\min(.3328 ( \tilde R + .429 ), .33554 ( \tilde R + .415 ))}{\sqrt{n}})
    \big).
\end{talign}
\end{remark}

\subsection{Proof sketch for \cref{efficient_tail,cpt_concentration}}\label{proof:sketch}
 Similarly to the proof of \cref{stein_thm_1} and \cref{eff_hellinger}, the main idea of the proofs is to use the fact that $S_n=\frac{1}{\sqrt{n}}\sum_{i\le n}Y_i$ is close to being normally distributed if the sample size $n\in \mathbb{N}$ is large. For ease of notation, the rest of this section we will suppose that $\sigma=1$. Note that there is no loss of generality by doing so. Fix $t \geq 0$, $\rho\in [0,1)$, and $p\geq 1$.
For all $\epsilon>0$ we remark that there is a coupling $(S_n,G)$ with $G\sim N(0,1)$ such that $\|S_n-G\|_p\le \wass (S_n,G)+\epsilon$.
 Therefore we have
\begin{align}
\label{eq:dino}
\Parg{S_n \geq t }
    &= \Parg{G + S_n-G \geq t }
  \\  &\leq \Parg{G \geq \rho t} + \Parg{S_n-G \geq (1-\rho) t} \\
	&\leq  \Phi^c({\rho t}) + \frac{\E((S_n - G)_+^p)}{(1-\rho)^pt^p} \\
	&\leq  \Phi^c({\rho t}) + \frac{(\wass (S_n,G)+\epsilon)^p}{(1-\rho)^pt^p}
\end{align}As this holds for any arbitrary $\epsilon>0$ we get that\begin{align}
\label{eq:dino}
\Parg{S_n \geq t }
    &= \Parg{G + S_n-G \geq t } \\
	&\leq  \Phi^c({\rho t}) + \frac{\wass (S_n,G)^p}{(1-\rho)^pt^p}.
\end{align} Imagine that $\wass (S_n,G)\le \frac{\tilde Kp}{\sqrt{n}}$ for a constant $\tilde K$ and that we set $p=\frac{\sqrt{n}t(1-\rho)}{e\tilde K}$ then we obtain
\begin{align}
\Parg{S_n \geq t }
	&\leq  \Phi^c({\rho t}) + e^{-\frac{\sqrt{n}t(1-\rho)}{e\tilde K}}.
\end{align}If in addition we choose
$$\rho\defeq  \frac{-C_n+\sqrt{C_n^2+2t^2(C_n+1-\log(C_n\sqrt{2\pi}/t))}}{t^2}$$ with $C_n\defeq \frac{\sqrt{n}t}{e\tilde K}$ then we have
\begin{align}
\P(S_n\ge t)&\lessapprox \Phi^c\big(\rho t\big)+\frac{e\tilde K}{\sqrt{n}}\varphi(\rho t);
\end{align}This gives us the desired bound.

\subsection{Proof sketch for \cref{efficient_quantile,cpt-quantile}}
Quantiles are obtained by following a similar reasoning. Let $\delta\in (0,\frac{1}{2})$ be a confidence level and $\alpha \in (0,\delta)$. Choose $\epsilon>0$ and $(S_n,G)$ to be a coupling of $S_n$ with a standard normal random variable that satisfies $\|S_n-G\|_p\le \wass (S_n,\mathcal{N}(0,1))+\epsilon$. Then for all $p\ge 2$, if we write $u\defeq \frac{\wass (S_n,G)}{\alpha^{1/p}}$ then we obtain
\begin{align}
    \P(S_n\ge -\Phi^{-1}(\delta-\alpha)+u)&\le \P(G\ge -\Phi^{-1}(\delta-\alpha))+\P(S_n-G\ge u)
    \\&\le \delta -\alpha +\frac{(\|S_n-G\|_p+\epsilon)^p}{u^p}.
\end{align}
As the choice of $\epsilon>0$ can be arbitrarily small we obtain
\begin{talign}
    \P(S_n\ge -\Phi^{-1}(\delta-\alpha)+u)&\le \delta-\alpha +\frac{\wass (S_n,G)^p}{u^p}
    \le \delta.
\end{talign}
Therefore to obtain a good approximation of the quantiles of $S_n$ we need to choose a candidate for $\alpha$ and $p$. If we choose $p=-\log(\alpha)$ then we obtain then we can upper bound the quantile of $S_n$ by $$ -\Phi^{-1}(\delta-\alpha)+e\mathcal{W}_{-\log(\alpha)}(S_n,G)\defeq O(-\Phi^{-1}(\delta-\alpha)+ \frac{-\log(\alpha)}{\sqrt{n}}).$$ We choose $\alpha=O(\frac{\varphi(\Phi^{-1}(\delta))}{\sqrt{n}})$.%

\section{Efficient Concentration Inequalities for Smoothed Observations}\label{smoothed}

We notice that when we had some additional information on the distribution of the random variables $(Y_i)$, e.g about their score function (see \cref{bound}) and their Stein kernel (see \cref{stein_thm_3}), then we  were able to avoid potentially lossy union bound and obtained tighter concentration inequalities.
 In this section we show that if the observations $(Y_i)$ are smoothed then we can obtain (potentially) tighter concentration inequalities without requiring any additional properties of the distribution of the random variables $(Y_i)$. In this goal let $D>0$ be a real and let $Z$ be an independent standard normal random variable. We define $$\tilde S_n^{\rm{noisy}}\defeq \sqrt{1-\frac{D}{\sqrt{n}}}\tilde S_n+\frac{\sqrt{D}}{n^{1/4}}Z,\qquad S_n^{\rm{noisy}}\defeq \sigma \tilde S_n^{\rm{noisy}}.$$ We show how we can obtain efficient tail bounds for $\tilde S_n^{\rm{noisy}}$. In this goal, we write $\omega_{1,p,q}=\mathcal{W}_{\frac{2pq}{q-1}}(\tilde  S_n,Z)$ and $\omega_{2,p,q}=\|Z\|_{\frac{pq}{q-1}}$.
\begin{theorem}[Efficient smoothed tail bound]\label{strss}
    Assume that the random variables $(Y_i)$ satisfy \cref{assump:bounded-dev}.  Let $\hat{Q}_n(u)$ be an arbitrary upper bound for $\P(\tilde S_n\ge u)\le \hat{Q}_n(u)$. Then the following holds for all $\kappa\ge \max(\frac{\tilde R^2}{n},\frac{\tilde R^2}{\sqrt{n}D})$
    \begin{align}&
        \P( S_n^{\rm{noisy}}\ge \sigma u)
\\&\lessapprox\Phi^c(u)+\inf_{\substack{p\ge 2}}\frac{\hat{Q}_n(u)^{\frac{p-1}{p}}}{\sqrt{n}}\Big\{\frac{{\sqrt{e}(p-1)\tilde R^2}(e^{2\kappa(p-1)}-1)}{\sqrt{3}D\kappa }+\frac{\log(n/D^2)\sqrt{p-1}A_p}{4\sqrt{2}} (\tilde R-1)\Big\}%
    \end{align}
\end{theorem}
\begin{remark}
    The exact bound is presented in \cref{obs}. We also note that this bound could not be obtained by directly applying \cref{bound} or \cref{stein_thm_3}. Indeed while the score function $\tilde S_n^{\rm{noisy}}$ can be seen as an empirical average of continuous random variables the amount of Gaussian noise injected is not sufficient to control its score function or its Stein kernel.
\end{remark}Note that one can straightforwardly bound the Wassertein distance between $S_n^{\rm{noisy}}$ and $S_n$ and use this to obtain an efficient concentration inequality for the latter. Similarly if $S_n$ admits a Stein kernel then one could use strong embeddings \citet{chatterjee2012new} (Theorem 1.2) to control the tail bound of $|S_n-S_n^{\rm{noisy}}|$ and use \cref{strss} to obtain an efficient concentration inequality for $S_n$.
\subsection{Proof sketch for  \cref{strss}}
The proof relies on the Ornstein-Ulbheck interpolation and the heat equation. To make this precise we write the interpolated process $X_t\defeq e^{-t}\tilde S_n+\sqrt{1-e^{-2t}}Z$. As $t$ varies from $0$ to $\infty$, then $X_t$ transforms from $\tilde S_n$ to $Z$. We define $T_n\defeq-\frac{1}{2}\log(1-\frac{D}{\sqrt{n}})$. We note that $X_{T_n}$ has the same distribution than $\tilde S_n^{\rm{noisy}}$. The key to the proof is then to use the heat equation to establish the following equality:
\begin{align}\Big|\P(\tilde S_n^{\rm{noisy}}\ge u)-\Phi^c(u)\Big|&=\Big|\int_{T_n}^{\infty}\mathbb{E}\Big[\mathbb{I}(X_t\ge u)\mathbb{E}\Big[\frac{e^{-2t}}{{1-e^{-2t}}}(Z^2-1)-\frac{e^{-t}}{\sqrt{1-e^{-2t}}}\tilde S_nZ-\tau'_t|X_t\Big]\Big] dt\Big|
\end{align} where $(\tau'_t)$ is the quantity defined in \cref{tyl}. The proof is finished by using H\"{o}lder's inequality and bounding the norms $\|\frac{e^{-2t}}{{1-e^{-2t}}}(Z^2-1)-\frac{e^{-t}}{\sqrt{1-e^{-2t}}}\tilde S_nZ-\tau'_t\|_p$ (see \cref{ahahhhhhh} and \cref{aaaa2}).

\section{Application: Efficient \emph{Empirical} Berry-Esseen Bounds}\label{empi}
The preceding sections assumed that the variance parameter $\sigma^2$ was a known quantity, but, in many applications, the variance is unknown and can only be estimated from data.
In this section, we leverage efficient known-variance bounds to develop efficient quantile bounds that are valid even when $\sigma$ is unknown.
We refer to these constructions as   \emph{empirical Berry-Esseen} (EBE) bounds as they combine Gaussian approximation in the spirit of Berry-Esseen \cref{berryesseen} with empirical variance estimation.

We begin by showing how to convert generic known-variance quantile bounds into valid empirical-variance quantile bounds.
Our construction in \cref{unknown} depends on the following readily-computed quantities:
\begin{itemize}
    \item the empirical variance: $\hat\sigma^2 \defeq \frac{1}{n}\sum_{i\le n}(W_i-\bar{W}_n)^2$,
    \item the empirical centered third moment: $\hat m_3\defeq \frac{1}{n}\sum_{i\le n}(W_i-\bar{W}_n)^3$,
    \item the empirical variance of $(W_i - \bar{W}_n)^2$: $\hat{m}_4\defeq \frac{1}{n}\sum_{i\le n}((W_i-\bar{W}_n)^2-\hat\sigma^2)^2$,
    \item the $1-\frac{a}{4}$ quantile of the two-sided Kolmogorov-Smirnov distribution: $q_n^{\rm{KS}}(a/4)$,
    \item a lower estimate for $\sigma^2$:  $\sigma_{\rm{l},a}^2\defeq \hat \sigma^2-R^2q_n^{\rm{KS}}(\frac{a}{4})$,
    \qtext{and}
    \item a lower estimate for $s_4^2=\Var((W_1-\E(W_1))^2)$:
    \begin{talign}
    s_{\rm{l},a}^2\defeq \hat{m}_4 - R^4q_n^{\rm{KS}}(\frac{a}{4}) + \min_{|u| \leq 2Rq_n^{\rm{KS}}(\frac{a}{4})} 4\hat\sigma^2 u^2 - 4 \hat{m}_3 u,
    \end{talign}
\end{itemize}
and the proof can be found in \cref{sec:proof_of_unknown}.

\begin{theorem}[Empirical quantile bounds]\label{unknown}
Consider any quantile bounds $\twoq$ and $\oneq$  with
\begin{talign}
P\big(|S_n|\geq \twoq(R,\delta,\sigma)\big)\leq \delta
&\qtext{and}
c^{\delta}_1(\sigma) \defeq \frac{\twoq(R,\delta,\sigma)^2}{n\sigma^2}
\qtext{non-increasing}
\qtext{and}\\
P\big(S_n> \oneq(R,\delta,\sigma)\big) \leq \delta
&\qtext{and}
c^{\delta}_2(\sigma) \defeq \frac{\oneq(R^2,\delta,\sigma)^2}{n\sigma^2}
\qtext{non-increasing}
\end{talign}
for each $\delta\in(0,1)$ and $S_n$ satisfying \cref{assump:bounded-dev}.
If $S_n$ satisfies \cref{assump:bounded-dev}, then, for each confidence level $\delta\in(0,1)$ and $a_n \in (0,\delta)$, we have
$$P\big(S_n \notin \sqrt{n} A^n_{\delta,a_n}\big)\le \delta
\qtext{and hence}
P\big(|S_n| \geq \hat{q}^{d}_n(R,\delta,a_n) \big)\le \delta
$$
for the empirical quantile bound $\hat{q}^{d}_n(R,\delta,a_n) \defeq
\sqrt{n} \max_{x\in A^n_{\delta,a_n}} |x|$
and the confidence set
\begin{talign}
       A^n_{\delta,a}\defeq&\Big\{x\in\mathbb{R}\mid \Big(x^2 - \hat\sigma^2\frac{c^{\delta-a}_1(\sigma_{\rm{l},a})}{1-c^{\delta-a}_1(\sigma_{\rm{l},a})} \Big)^2
\\&\le  \Big(\frac{c^{\delta-a}_1(\sigma_{\rm{l},a})}{1-c^{\delta-a}_1(\sigma_{\rm{l},a})}  \Big)^2 \frac{c^a_2(s_{\rm{l},a})}{1-c^a_2(s_{\rm{l},a})} \Big[ \hat{m}_4 + 4\hat\sigma^2x^2 - 4\hat m_3 x +  R^4 q_n^{\rm{KS}}(\frac{a}{4})\Big]\Big\}.
   \end{talign}
\end{theorem}
\begin{remark}
The maximum and minimum elements of $A^n_{\delta,a_n}$ are roots of a depressed quartic equation that can be solved in closed form or with standard software.
\end{remark}

Next we prove that  efficient quantile bounds give rise to  efficient empirical quantile bounds.
\begin{theorem}[Efficient empirical Berry-Esseen bounds]\label{efficient-unknown}
Under the assumptions of \cref{unknown}, suppose
$\twoq(R,\sigma_n,\delta_n)\to -\sigma\Phi^{-1}(\frac{\delta}{2})$
whenever $(\sigma_n, \delta_n)\to (\sigma, \delta)$. If
$a_n \to 0$,
$\frac{\log(a_n)}{n} \to 0$,
and
$c_2^{a_n}(s_{\mathrm{l},a_n}) \stackrel{a.s.}{\to} 0$,
then
\begin{talign}
\hat{q}^{d}_n(R,\delta,a_n)
\stackrel{a.s.}{\to} -\sigma\Phi^{-1}(\frac{\delta}{2}).
\end{talign}
\end{theorem}
\begin{remark}\label{default-efficient-unknown}
As convenient default choices, we recommend setting $a_n = \frac{\log n}{n}$ and taking $\twoq$ and $\oneq$ to be the efficient quantile bounds $q^d_n,q_n$ of \cref{cpt-quantile} with the auxiliary bounds of \cref{aux_q_bounds}.
\end{remark}
\begin{proof}
For brevity, we will adopt the shorthand $\hat{q}^{d}_n \defeq \hat{q}^{d}_n(R,\delta,a_n)$ and $\twoq \defeq \twoq(R,\delta-a_n,\sigma_{\mathrm{l},a_n})$.
Since $\sqrt{n}A^n_{\delta,a_n}$ is the sublevel set of the norm-coercive monic polynomial
\begin{talign}\label{eq:emp_polynomial}
y\mapsto \Big(y^2 - \frac{\hat\sigma^2}{\sigma_{\mathrm{l},a_n}^2}\frac{\twoq}{1-c^{\delta-a_n}_1(\sigma_{\mathrm{l},a_n})} \Big)^2
-
\Big(\frac{\twoq/\sigma_{\mathrm{l},a_n}^2}{1-c^{\delta-a_n}_1(\sigma_{\mathrm{l},a_n})}  \Big)^2 \ \frac{c^{a_n}_2(s_{\mathrm{l},a_n})(\hat{m}_4 + \frac{4\hat\sigma^2y^2}{n} - \frac{4\hat m_3}{\sqrt{n}} +  R^4 q_n^{\rm{KS}}(\frac{a_n}{4}))}{1-c^{a_n}_2(s_{\mathrm{l},a_n})},
\end{talign}
$\hat{q}^{d}_n$ is necessarily a root of this polynomial.

Moreover, the coefficients of this polynomial converge as follows.
First, the Dvoretsky-Kiefer-Wolfowitz inequality \citep{dvoretsky1956} implies that
\begin{talign}
q_n^{\rm{KS}}(\frac{a_n}{4})
    \leq \sqrt{\frac{\log(C/a_n)}{n}}
\end{talign}
for some constant $C$.
Our assumption $\frac{\log(a_n)}{n} \to 0$ therefore yields $q_n^{\rm{KS}}(\frac{a_n}{4})\to 0$.
Moreover, by the strong law of large numbers \citep[Thm.~2.4.1]{durrett2019probability}, $\hat\sigma^2 \stackrel{a.s.}{\to} \sigma^2$, $\hat m_4\stackrel{a.s.}{\to} s_4^2$, and $\hat m_3\stackrel{a.s.}{\to} \E[(W_i - \E W_1)^3]$ and hence $\sigma_{\rm{l},a}^2\stackrel{a.s.}{\to} \sigma^2$ and $s_{\rm{l},a}^2 \stackrel{a.s.}{\to} s_4^2$ as well.
Since $a_n\to 0$, we therefore have $\twoq\stackrel{a.s.}{\to} -\sigma\Phi^{-1}(\frac{\delta}{2})$ and $c^{\delta-a_n}_1(\sigma_{\mathrm{l},a_n})\stackrel{a.s.}{\to} 0$.
Finally, as $c_2^{a_n}(s_{\mathrm{l},a_n}) \to 0$, the polynomial \cref{eq:emp_polynomial} converges almost surely to the monic polynomial $y \mapsto (y^2 - (-\sigma\Phi^{-1}(\frac{\delta}{2})))^2$.
Hence, by \citet[Prop.~5.2.1]{artin2011algebra}, $\hat{q}^{d}_n$  converges almost surely to the unique positive root of this polynomial, $-\sigma\Phi^{-1}(\frac{\delta}{2})$.
\end{proof}

\subsection{Numerical evaluation}\label{sec:numerical}
We next turn to a numerical evaluation of our efficient EBE bound. Python code implementing our bound and reproducing all plots can be found at \url{https://github.com/lmackey/gauss_conc/}.
\cref{fig:emp_quantile} compares the efficient EBE bound of \cref{efficient-unknown,default-efficient-unknown} with the Hoeffding quantile bound,
$\P(|S_n| \geq R\sqrt{2  \log(2/\delta)}) \leq \delta$;
the
empirical Bernstein quantile bound \citep[Thm.~4]{maurer2009empirical} commonly deployed in reinforcement learning \citep{mnih2008empirical,audibert2009exploration}, $\P(|S_n| \geq \hat{\sigma} \sqrt{2  \log(2/\delta) \frac{n}{n-1}} + \frac{14}{3} R\log(2/\delta) \frac{\sqrt{n}}{n-1}) \leq \delta$; and the efficient $I_{n,3}$ quantile bound of \citet{romano2000finite},\footnote{The Romano-Wolf bound can be infinite, but we constrain it to be no larger than the always-valid Hoeffding bound.}
with its free parameter $\beta_n$ set to match our default value, $a_n = \frac{\log n}{n}$.

As advertised, the efficient bounds converge to the optimal asymptotic size as $n$ increases, while the Bernstein and Hoeffding inequalities remain bounded away from the optimum for all $n$.
Moreover, the efficient EBE bound of \cref{unknown} provides a tighter estimate than the optimized Romano-Wolf bound in all settings.
Here the efficient empirical bound is constructed with $\hat m_3 = R \hat{\sigma}^2$ and $\hat m_4 = R^2 \hat{\sigma}^2 - \hat{\sigma}^4$ and, as we show in \cref{additional_plots}, changes only minimally as $\hat m_3$ and $\hat m_4$ vary.

\begin{figure}[htb!]
\floatconts
  {fig:emp_quantile}
  {\caption{Empirical quantile bounds for $|S_n|$ with fixed boundedness parameter $R=1$ and varying sample size $n$ and empirical variance $\hat{\sigma}^2$. Unlike the empirical Bernstein and Hoeffding bounds, the efficient empirical Berry-Esseen bound of \cref{unknown} converges to the ideal asymptotic Gaussian bound as $n$ increases and provides a tighter estimate than the efficient $I_{n,3}$ interval of \citet{romano2000finite} in all settings.}}
  {
    \includegraphics[width=\textwidth]{emp_quantile_bounds-two_sidedTrue-v3x1-v4x1.pdf}

  }
\end{figure}

\section{Discussion and Related Work}
In this work, we have derived new computable tail and quantile bounds for the scaled deviations $S_n = \sqrt{n}(\bar{W}_n - \E(W_1))$ with asymptotically optimal size, finite-sample validity, and sub-Gaussian decay.
These bounds enable the construction of efficient confidence intervals with correct coverage for any sample size.
Our concentration inequalities arise from new computable bounds on the Hellinger distance, Stein discrepancy, and Wasserstein distance to a Gaussian, and, as a byproduct, we obtain the first explicit bounds for the Hellinger CLT.  We also develop a general method for bounding the Hellinger distance between a Gaussian and any other random variable.

The notion of efficient confidence intervals for the mean was introduced by \citet{romano2000finite}. In their Thm.~2.1, \citeauthor{romano2000finite}  showed that efficient---that is, asymptotically minimal-length when scaled by $\sqrt{n}$---confidence intervals must converge to the width of the asymptotic Gaussian intervals implied by the CLT \cref{eq:gaussian_tails}.
Moreover, Sec.~3 of \citeauthor{romano2000finite} surveys a number of procedures for constructing confidence intervals that are either finite-sample invalid (including the bootstrap \citep{efron1992bootstrap} and methods based on Edgeworth expansions \citep{hall2013bootstrap}) or inefficient (including the methods of \citet{anderson1969confidence} and \citet{gasko1991new}).
\citeauthor{romano2000finite} conclude by developing an efficient valid confidence interval for the mean of variables supported on $[0,1]$ but report that it is ``unfortunately, much too wide for a reasonable sample.''
Our new efficient bounds are developed in an entirely different manner, and, as we  demonstrate in \cref{sec:numerical}, improve upon the Romano-Wolf interval and the most commonly used empirical concentration inequalities.

Our results also suggest simple strategies for making any concentration inequality or confidence region efficient. For efficient concentration, one can simply take the minimum of any existing tail bound and  \cref{cpt_concentration} to simultaneously reap the small-sample benefits of the former and the large-sample benefits of the latter.
For efficient confidence, one can divide the total confidence budget between an existing region and the efficient region of \cref{efficient-unknown} and then intersect the two regions.
The result will remain efficient if the budget allocated to the auxiliary region vanishes as $n$ grows.
These strategies are particularly relevant given the recent renewed interest in deriving tighter concentration inequalities for bounded random variables \citep[see, e.g.,][]{jun2019parameter,waudby2020estimating,orabona2021tight}.

Our Hellinger bound is, to the best of our knowledge, the first known explicit rate of convergence for the central limit theorem in terms of the Hellinger distance. Indeed while rates were known for the KL divergence, none were explicitly derived for the Hellinger distance. Moreover, the KL rates are neither explicit nor computable and require stronger conditions. However, our proof technique builds on previously developed ideas. Notably, we use the Ornstein-Ulenbeck semigroup to interpolate between $\tilde S_n$ and a normal. This allows us to re-express the Hellinger distance as an integral over the norm of information-theoretic quantities of the interpolated process. This can be seen as the equivalent of the De Bruijn identity (see e.g \citep{barron1986entropy}) for the KL-divergence or similar expressions  obtained for $p$-Wassertein distance in the celebrated work of  \citet{otto2000generalization} and in \citet{ledoux2015stein}. Then we take inspiration from \citet{bonis2015rates} which proposed to use an exchangeable pair argument to give a closed form upper bound to the $p$-Wassertein distance and develop similar techniques.  This provides us with an easily computable bound that only depends on the moments of the score function and the moments of the random variable itself. This is to contrast with the known bounds for the KL divergence, which are either not in closed form \citep{bobkov2013rate}, require conditions on the spectral gap \citep{artstein2004rate} or depend on the moment of the Stein kernel \citep{ledoux2015stein}.

The notion of a Stein kernel (sometimes known under the name of \emph{Stein factor}, \emph{$\omega$-function}, or \emph{covariance kernel} \citep[Rem.~4.6]{ley2017stein}) is born from the literature of Stein's approximation and is found implicitly in many of the earlier works (see for e.g the original monograph \citep{stein1986approximate}). It was however first formally introduced in \citet{cacoullos1992lower} and  has been the subject of a lot of recent development including to establish multivariate central limit theorems \citep{nourdin2010multivariate}, to compare univariate distributions \citep{ley2017stein} for its connection with the total variation distance \citep{cacoullos1994variational}, the $p$-Wassertein distance \citep{ledoux2015stein,bonis2015rates}, the entropic central limit theorem \citep{nourdin2014entropy,ledoux2015stein}, covariance bounds \citep{ernst2020first} and concentration inequalities \citep{ledoux2015stein,nourdin2009density,saumard2019weighted}. Our theorem \cref{stein_thm_1} takes inspiration from the concentration inequalities obtained in \citet{ledoux2015stein}, but the inequality that we obtain is efficient and is on the tail of the $S_n$ rather than on the moments of Lipcthiz functions of $S_n$. The Stein kernel is however sadly not always guaranteed to exist. This led to a large body of work investigating under what conditions such a quantity exists and how to compute it ( see e.g \citep{courtade2017existence,fathi2019stein}).

Our Wasserstein-bounding arguments in \cref{simple} build on the pioneering work of \citet{bonis2015rates} who derived Wasserstein convergence rates for the CLT with inexplicit constants. Our new arguments lead to tighter estimates of the distance to Gaussianity  and explicit, practical constants.

An important open question is if a similar approach could be used for other statistics, beyond sample means, that are known to be asymptotically normal. Another natural direction for future work is to relax the assumption than the random variables are bounded to conditions on the moments of $(Y_i).$
\newpage\clearpage

Probability (math.PR); Statistics Theory (math.ST)