--- abstract: 'We investigate the frequentist coverage of Bayesian credible sets in a nonparametric setting. We consider a scale of priors of varying regularity and choose the regularity by an empirical Bayes method. Next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity. We show that such an adaptive Bayes credible set gives correct uncertainty quantification of "polished tail" parameters, in the sense of high probability of coverage of such parameters. On the negative side, we show by theory and example that adaptation of the prior necessarily leads to gross and haphazard uncertainty quantification for some true parameters that are still within the hyperrectangle regularity scale.' address: - | B. Szabó\ Department of Mathematics\ Eindhoven University of Technology\ P.O. Box 513\ 5600 MB Eindhoven\ The Netherlands\ - | A. van der Vaart\ Mathematical Institute\ Leiden University\ P.O. Box 9512\ 2300 RA Leiden\ The Netherlands\ - | H. van Zanten\ Korteweg-de Vries Institute for Mathematics\ University of Amsterdam\ P.O. Box 94248\ 1090 GE Amsterdam\ The Netherlands\ author: - - - title: Frequentist coverage of adaptive nonparametric Bayesian credible sets --- ./style/arxiv-general.cfg , \[class=AMS\] Introduction {#sec1} ============ In Bayesian nonparametrics posterior distributions for functional parameters are often visualized by plotting a center of the posterior distribution, for instance, the posterior mean or mode, together with upper and lower bounds indicating a *credible set*, that is, a set that contains a large fraction of the posterior mass (typically 95%). The credible bounds are intended to visualize the remaining uncertainty in the estimate. In this paper we study the validity of such bounds from a frequentist perspective in the case of priors that are made to adapt to unknown regularity. It is well known that in infinite-dimensional models Bayesian credible sets are not automatically frequentist confidence sets, in the sense that under the assumption that the data are in actual fact generated by a "true parameter," it is not automatically true that they contain that truth with probability at least the credible level. The earliest literature focused on negative examples, showing that for many combinations of truths and priors, Bayesian credible sets can have very bad or at least misleading frequentist behavior; see, for instance, @Cox, @Freedman, @Johnstone. \[An exception is @Wahba, who showed encouraging simulation results and gives heuristic arguments for good performance.\] However, credible sets do not always have bad frequentist coverage. In the papers @Leahu, @Bartek ([-@Bartek; -@Bartek2]) this matter was investigated in the setting of the canonical (inverse) signal-in-white-noise model, where, essentially, the unknown parameter was a function with a fixed regularity and the (Gaussian) prior had a fixed regularity as well. The main message in these papers is that Bayesian credible sets typically have <span>*good*</span> frequentist coverage in case of undersmoothing (using a prior, i.e., less regular than the truth), but can have coverage zero and be far too small in the other case. Simulation studies corroborate these theoretical findings and show that the problem of misleading uncertainty quantification is a very practical one. The solution to undersmooth the truth, which gives good uncertainty quantification, is unattractive for two reasons. First, it leads to a loss in the quality of the reconstruction, for example, by the posterior mode or mean. Second, the true regularity of the functional parameter is never known and hence cannot be used to select a prior that undersmoothes the right regularity. Therefore, in practice, it is common to try and "estimate" the regularity from the data, and thus to <span>*adapt*</span> the method to the unknown regularity. Bayesian versions of this approach can be implemented using empirical or hierarchical Bayes methods. Empirical Bayes methods estimate the unknown regularity using the marginal likelihood for the data in the Bayesian setup; see Section \[SectionEB\] for a precise description. Hierarchical Bayes methods equip the regularity parameter with a prior and follow a full Bayesian approach. In the present paper we concentrate on the empirical Bayes approach. In the context of the inverse signal-in-white-noise model, this method has been shown to be rate-adaptive, in the sense that the posterior contracts at a (near) optimal rate around the truth for a range of true regularities, without using information about this regularity \[see @KSzVZ for an analysis of the method in the present paper or @Ray for similar work\]. However, these papers only address contraction of the posterior and do not investigate frequentist coverage of credible sets, which is perhaps more important than rate-adaptiveness to validate the use of these methods. In the present paper we study whether the empirical Bayes method, which is optimal from the point of view of contraction rates, also performs well from the perspective of coverage. In particular, we investigate to which extent the method yields adaptive confidence sets. Bayesian credible sets can of course not beat the general fundamental limitations of adaptive confidence sets. As pointed out by @Low, it is in general not possible to construct confidence sets that achieve good coverage across a range of nested models with varying regularities and at the same time possess a size of optimal order when the truth is assumed to be in one of the particular sub-models. Similar statements, in various contexts, can be found in @JudLam, @CaiLow ([-@CaiLow; -@CaiLow2006]), @RobVaart, @GenWas, and @Hoff. We show in this paper that for the standard empirical Bayes procedure (which is rate-adaptive), there always exist truths that are not covered asymptotically by its credible sets. This bad news is alleviated by the fact that there are only a few of these "inconvenient truths" in some sense. For instance, the minimax rate of estimation does not improve after removing them from the model; they form a small set in an appropriate topological sense; and they are unlikely under any of the priors. The good news is that after removing these bad truths, the empirical Bayes credible sets become adaptive confidence sets with good coverage. Our results are inspired by recent (non-Bayesian) results of @GineNickl and @Bull. These authors also remove a "small" set of undesirable truths from the model and focus on so-called self-similar truths. Whereas these papers use theoretical frequentist methods of adaptation, in the present paper our starting point is the Bayesian (rate-adaptive) procedure. This generates candidate confidence sets for the true parameter (the credible sets), that are routinely used in practice. We next ask for which truths this practice can be justified and for which not. Self-similar truths, defined appropriately in our setup, are covered, but also a more general class of parameters, which we call *polished tail sequences*. The paper is structured as follows. In the next section we describe the setting: the inverse signal-in-white-noise model and the adaptive empirical Bayes procedure. In Section \[secmain\] the associated credible sets are constructed and analyzed. A first theorem exhibits truths that are not covered asymptotically by these sets. The second theorem shows that when these "inconvenient truths" are removed, the credible sets yield adaptive, honest confidence sets. The theoretical results are illustrated in Section \[secsim\] by a simulation study. Proofs are given in Sections \[secproofcov\]–\[SectionTechnicalLemmas\]. In Section \[seccon\] we conclude with some remarks about possible extensions and generalizations. Finally, the is a self-contained proof of a version of an important auxiliary result first proved in @KSzVZ. We conclude the by further discussions of adaptive nonparametric confidence sets and the coverage of credible sets. The credible sets we consider in this paper are $\ell_2$-balls, even though we believe that similar conclusions will be true for sets of different shapes. It is known that $\ell_2$-confidence balls can be honest over a model of regularity ${\alpha}$ and possess a radius that adapts to the minimax rate whenever the true parameter is of smoothness contained in the interval $[{\alpha}, 2{\alpha}]$. Thus, these balls can adapt to double a coarsest smoothness level \[@JudLam, @CaiLow2006, @RobVaart, @BullNickl\]. The fact that the coarsest level ${\alpha}$ and the radius of the ball must be known \[as shown in @BullNickl\] makes this type of adaptation somewhat theoretical. This type of adaptation is not considered in the present paper (in fact, we do not have a coarsest regularity level ${\alpha}$). Work we carried out subsequent to the present paper \[@SzaboJSPI\] indicates that this type of adaptation can be incorporated in the Bayesian framework, but requires a different empirical Bayes procedure as the one in the present paper \[based on the likelihood (\[EqDefinitionHata\])\]. Interestingly, with the latter method and for known ${\alpha}$, adaptation occurs for all true parameters, also the inconvenient ones. The credible sets considered in the present paper result from posterior distributions for infinite-dimensional parameters, or functions, which implicitly make the bias–variance trade-off that is characteristic of nonparametric estimation. These posterior distributions induce marginal posterior distributions for real-valued functionals of the parameter. If such a functional is sufficiently smooth, then the corresponding marginal posterior distribution may satisfy a Bernstein–von Mises theorem, which typically entails that the bias is negligible relative to the variance of estimation. Just as in the case of finite-dimensional models, such an approximation implies that the credible sets for the functional are asymptotically equivalent to frequentist confidence sets. In this sense nonparametric priors and posteriors may yield exact, valid credible sets. By extending this principle to an (infinite) collection of smooth functionals that identifies the parameter, @CastilloNickl even obtain an exact credible set for the full parameter. However, elegant as their construction may be, it seems that no method that avoids dealing with the bias–variance trade-off will properly quantify the uncertainty of nonparametric Bayesian inference as it is applied in current practice. Notation {#SectionNotation} -------- The $\ell^2$-norm of an element ${\theta}\in\ell^2$ is denoted by ${\Vert}{\theta}{\Vert}$, that is, ${\Vert}{\theta}{\Vert}^2=\sum_{i=1}^{\infty}{\theta}_i^2$. The *hyperrectangle* and *Sobolev space* of order ${\beta}>0$ and (square) radius $M > 0$ are the sets $$\begin{aligned} \Theta^{{\beta}}(M)&=&\Bigl\{{\theta}\in\ell^2\dvtx \sup _{i\ge1}i^{1+2{\beta}}{\theta}_i^2 \le M \Bigr\},\label{defhyperrectangle} \\ S^{{\beta}}(M)&=&\Biggl\{{\theta}\in\ell^2\dvtx \sum _{i=1}^\infty i^{2{\beta}}{\theta}_i^2 \le M\Biggr\}.\label{defsobolevspace}\end{aligned}$$ For two sequences $(a_n)$ and $(b_n)$ of numbers, $a_n \asymp b_n$ means that ${\vert}a_n/b_n{\vert}$ is bounded away from zero and infinity, $a_n \lesssim b_n$ that $a_n/b_n$ is bounded, $a_n \sim b_n$ that $a_n/b_n \rightarrow1$, and $a_n \ll b_n$ that $a_n / b_n \rightarrow0$, all as $n$ tends to infinity. The maximum and minimum of two real numbers $a$ and $b$ are denoted by $a \vee b$ and $a \wedge b$. Statistical model and adaptive empirical Bayes procedure {#SectionEB} ======================================================== We formulate and prove our results in the canonical setting of the inverse signal-in-white-noise model. As usual, we reduce it to the sequence formulation. See, for instance, @Cavalier and the references therein for more background and many examples fitting this framework. The observation is a sequence $X=(X_1,X_2,\ldots)$ satisfying $$\label{eqdata} X_i={\kappa}_i{\theta}_{0,i}+ \frac{1}{\sqrt{n}}Z_i, \qquad i = 1, 2, \ldots,$$ where ${\theta}_{0}=({\theta}_{0,1},{\theta}_{0,2},\ldots)\in\ell^2$ is the unknown parameter of interest, the ${\kappa}_i$'s are known constants (transforming the truth) and the $Z_i$ are independent, standard normally distributed random variables. The rate of decay of the ${\kappa}_i$'s determines the difficulty of the statistical problem of recovering ${\theta}_0$. We consider the so-called mildly ill-posed case where $$C^{-2}i^{-2p}\leq{\kappa}_i^2\leq C^2 i^{-2p}\label{assumpinverse},$$ for some fixed $p \ge0$ and $C > 0$. In particular, the choice $p=0$ corresponds to the ordinary signal-in-white-noise model, whereas $p>0$ gives a true *inverse problem*. For ${\alpha}> 0$ we define a prior measure $\Pi_{\alpha}$ for the parameter ${\theta}_0$ in (\[eqdata\]) by $$\label{prior} \Pi_{{\alpha}}=\bigotimes_{i=1}^{\infty}N \bigl(0,i^{-1-2{\alpha}}\bigr).$$ The coordinates ${\theta}_i$ are independent under this prior. Since the corresponding coordinates of the data are also independent, the independence is retained in the posterior distribution, which by univariate conjugate Gaussian calculation can be seen to be $$\label{PostDist} \Pi_{\alpha}(\cdot\mid X) = \bigotimes _{i=1}^{\infty}{N} \biggl(\frac {n{\kappa}_i^{-1}}{i^{1+2{\alpha}}{\kappa}_i^{-2} + n}X_i, \frac{{\kappa}_i^{-2}}{i^{1+2{\alpha}}{\kappa}_i^{-2}+n} \biggr).$$ The prior (\[prior\]) puts mass 1 on Sobolev spaces and hyperrectangles of every order strictly smaller than ${\alpha}$ (see Section \[SectionNotation\] for definitions), and hence expresses a prior belief that the parameter is regular of order (approximately) ${\alpha}$. Indeed, it is shown in @Bartek that if the true parameter ${\theta}_0$ in (\[eqdata\]) belongs to a Sobolev space of order ${\alpha}$, then the posterior distribution contracts to the true parameter at the minimax rate $n^{-2{\alpha}/(1+2{\alpha}+2p)}$ for this Sobolev space. A similar result can be obtained for hyperrectangles. On the other hand, if the regularity of the true parameter is different from ${\alpha}$, then the contraction can be much slower than the minimax rate. The suboptimality in the case the true regularity is unknown can be overcome by a data-driven choice of ${\alpha}$. The *empirical Bayes* procedure consists in replacing the fixed regularity ${\alpha}$ in (\[PostDist\]) by (for given $\Aupper$, possibly dependent on $n$) $$\label{EqDefinitionHata} \hat{\alpha}_n={\mathop{\operatorname{argmax}}}_{{\alpha}\in[0,\Aupper]} \ell_n({\alpha}),$$ where $\ell_n$ is the marginal log-likelihood for ${\alpha}$ in the Bayesian setting: ${\theta}\mid{\alpha}\sim\Pi_{\alpha}$ and $X\mid({\theta}, {\alpha})\sim\bigotimes_{i}{N}({\kappa}_i{\theta}_i,1/n)$. This is given by $$\ell_n({\alpha})=-\frac{1}{2}\sum_{i=1}^{\infty} \biggl( \log\biggl(1+\frac{n}{i^{1+2{\alpha}}{\kappa}_i^{-2}} \biggr)-\frac{n^2} {i^{1+2{\alpha}}{\kappa}_i^{-2}+n}X_i^2 \biggr).\label{fnalpha}$$ If there exist multiple maxima, any one of them can be chosen. The *empirical Bayes posterior* is defined as the random measure $\Pi_{\hat{\alpha}_n}(\cdot\mid X)$ obtained by substituting $\hat{\alpha}_n$ for ${\alpha}$ in the posterior distribution (\[PostDist\]), that is, $$\label{EqEBPosteriorDistribution} \Pi_{\hat{\alpha}_n}(\cdot\mid X) = \Pi_{{\alpha}}(\cdot\mid X) |_{{\alpha}= \hat{\alpha}_n}.$$ In @KSzVZ this distribution is shown to contract to the true parameter at the (near) minimax rate within the setting of Sobolev balls and also at the near optimal rate in situations of supersmooth parameters. Extension of their results shows that the posterior distribution also performs well for many other models, including hyperrectangles. Thus, the empirical Bayes posterior distribution manages to recover the true parameter by *adapting* to unknown models. We now turn to the main question of the paper: can the spread of the empirical Bayes posterior distribution be used as a measure of the remaining uncertainty in this recovery? Notational assumption --------------------- We shall from now on assume that the first coordinate ${\theta}_{0,1}$ of the parameter ${\theta}_0$ is zero. Because the prior (\[prior\]) induces a $N(0,1)$ prior on ${\theta}_{0,1}$, which is independent of ${\alpha}$, the marginal likelihood function (\[fnalpha\]) depends on $X_1$ only through a vertical shift, independent of ${\alpha}$. Consequently, the estimator $\hat{\alpha}_n$ does not take the value of ${\theta}_{0,1}$ into account. While this did not cause problems for the minimax adaptivity mentioned previously, this does hamper the performance of credible sets obtained from the empirical Bayes posterior distribution, regarding uniformity in the parameter. \[In fact, the posterior distribution for ${\theta}_1$ has mean equal to $n\kappa_1^{-1}/(\kappa_1^{-2}+n)X_1$, independent of ${\alpha}$. The bias of this estimator of ${\theta}_{0,1}$ for fixed $n$ would lead to arbitrarily small coverage for values ${\theta}_{0,1}\rightarrow\pm\infty$, invalidating the main result of the paper, Theorem \[thmCoverage\] below.\] One solution would be to use the variances $(i+1)^{-1-2{\alpha}}$ in (\[prior\]). For notational simplicity we shall instead assume that ${\theta}_{0,1}=0$ throughout the remainder of the paper. Main results: Asymptotic behavior of credible sets {#secmain} ================================================== For fixed ${\alpha}> 0$, let $\hat{\theta}_{n, {\alpha}}$ be the posterior mean corresponding to the prior $\Pi_{\alpha}$ \[see (\[PostDist\])\]. The centered posterior is a Gaussian measure that does not depend on the data and, hence, for $\gamma\in(0,1)$ there exists a deterministic radius $r_{n,\gamma}({\alpha})$ such that the ball around the posterior mean with this radius receives a fraction $1-\gamma$ of the posterior mass, that is, for ${\alpha}>0$, $$\label{eqradius} \Pi_{\alpha}\bigl({\theta}\dvtx {\Vert}{\theta}- \hat {\theta}_{n, {\alpha}}{\Vert}\le r_{n, \gamma}({\alpha}) \mid X \bigr) = 1-\gamma.$$ In the exceptional case that ${\alpha}=0$, we define the radius to be infinite. The empirical Bayes credible sets that we consider in this paper are the sets obtained by replacing the fixed regularity ${\alpha}$ by the data-driven choice $\hat{\alpha}_n$. Here we introduce some more flexibility by allowing the possibility of blowing up the balls by a factor $L$. For $L > 0$ we define $$\label{EqCredibleSet} \hat C_n(L) = \bigl\{{\theta}\in\ell^2\dvtx {\Vert}{\theta}- \hat{\theta}_{n, \hat{\alpha}_n}{\Vert}\le Lr_{n, \gamma}(\hat {\alpha}_n) \bigr\}.$$ By construction, $\Pi_{\hat{\alpha}_n}(\hat C_n(L) \mid X) \ge1-\gamma $ iff $L \ge1$. We are interested in the performance of the random sets $\hat C_n(L)$ as frequentist confidence sets. Ideally, we would like them to be *honest* in the sense that $$\inf_{{\theta}_0\in\Theta_0}P_{{\theta}_0} \bigl({\theta}_0 \in\hat C_n(L) \bigr) \ge1-\gamma,$$ for a model $\Theta_0$ that contains all parameters deemed possible. In particular, this model should contain parameters of all regularity levels. At the same time we would like the sets to be *adaptive*, in the sense that the radius of $\hat C_n(L)$ is (nearly) bounded by the optimal rate for a model of a given regularity level, whenever ${\theta}_0$ belongs to this model. As pointed out in the , this is too much to ask, as confidence sets with this property, Bayesian or non-Bayesian, do not exist. For the present procedure we can explicitly exhibit examples of "inconvenient truths" that are not qanderson@example.net. \[LemmaCounterExample\] For given $\beta, M>0$ and $1\le\rho_j\uparrow\infty$, and positive integers $n_j$ with $n_{j+1}\geq(2\rho_{j+1}^2)^{1+2\beta+2p}n_j$, define ${\theta}_0=({\theta}_{0,1},{\theta}_{0,2},\ldots)$ by $$\begin{aligned} {\theta}_{0,i}^2=\cases{ 0, &\quad if $\rho_j^{-1}n_j^{\afrac{1}{1+2{\beta}+2p}} \leq i<n_j^{\afrac {1}{1+2{\beta}+2p}}$, $j=1,2,\ldots,$ \vspace*{3pt}\cr 0, &\quad if $2n_j^{\afrac{1}{1+2{\beta}+2p}}\leq i\leq\rho_jn_j^{\afrac {1}{1+2{\beta}+2p}}$, $j=1,2,\ldots,$ \vspace*{3pt}\cr Mi^{-1-2\beta}, &\quad otherwise.}$$ Then the constant $M$ can be chosen such that $P_{{\theta}_0}({\theta}_0 \in \hat C_{n_j}(L_{n_j})) \rightarrow0$ as $j \rightarrow\infty$ for every $L_{n_j}\lesssim\sqrt{M} \rho _j^{(1+2p)/(8+8\beta+8p)}$. For the proof see Section \[secCounterExample\]. By construction, the (fixed) parameter ${\theta}_0$ defined in Theorem \[LemmaCounterExample\] belongs to the hyperrectangle $\Theta^{\beta}(M)$, and in this sense is "good," because of the "smooth" truth. However, it is an inconvenient truth, as it tricks the empirical Bayes procedure, making this choose the "wrong" regularity ${\alpha}$, for which the corresponding credible set does not cover ${\theta}_0$. The intuition behind this counterexample is that for a given sample size or noise level $n$ the empirical Bayes procedure, and any other statistical method, is able to judge the coordinates ${\theta}_{0,1},{\theta}_{0,2},\ldots$ only up to a certain effective dimension $N_n$, fluctuations in the higher coordinates being equally likely due to noise as to a nonzero signal. Now if $({\theta}_{0,1},\ldots, {\theta}_{0,N_n})$ does not resemble the infinite sequence $({\theta}_{0,1},{\theta}_{0,2},\ldots)$, then the empirical Bayes procedure will be tricked into choosing a smoothness $\hat{\alpha}_n$ that does not reflect the smoothness of the full sequence, and failure of coverage results. The particular example ${\theta}_0$ in Theorem \[LemmaCounterExample\] creates this situation by including "gaps" of 0-coordinates. If the effective dimension is at the end of a gap of such 0-coordinates, then the empirical Bayes procedure will conclude that ${\theta}_0$ is smoother than it really is, and make the credible set too narrow. A more technical explanation can be given in terms of a bias–variance trade-off. For the given truth in Theorem \[LemmaCounterExample\] the bias of the posterior mean is of "correct order" $n^{-{\beta}/(1+2{\beta}+ 2p)}$ corresponding to $\Theta^{\beta}(M)$, but (along the subsequence $n_j$) the spread of the posterior is of strictly smaller order. Thus, the posterior distribution is overconfident about its performance. See Section \[secCounterExample\] for details. Intuitively, it is not surprising that such bad behavior occurs, as nonparametric credible or confidence sets always necessarily extrapolate into aspects of the truth that are not visible in the data. Honest uncertainty quantification is only possible by a priori assumptions on those latter aspects. In the context of regularity, this may be achieved by "undersmoothing," for instance, by using a prior of fixed regularity smaller than the true regularity. Alternatively, we may change the notion of regularity and strive for honesty over different models. In the latter spirit we shall show that the empirical Bayes credible sets $\hat C_n(L)$ are honest over classes of "polished" truths. A parameter ${\theta}\in\ell^2$ satisfies the *polished tail condition* if, for fixed positive constants $L_0$, $N_0$ and $\rho\ge2$, $$\label{EqPolishedTail} \sum_{i=N}^{\infty} {\theta}_i^2\le L_0 \sum _{i=N}^{\rho N}{\theta}_i^2\qquad \forall N\ge N_0.$$ We denote by $\Theta_{pt}(L_0,N_0,\rho)$ the set of all polished tail sequences ${\theta}\in\ell^2$ for the given constants $L_0$, $N_0$ and $\rho$. As the constants $N_0$ and $\rho$ are fixed in most of the following (e.g., at $N_0=2$ and $\rho=2$), we also use the shorter $\Theta_{pt}(L_0)$. \[It would be possible to make a refined study with $N_0$ and $L_0$ tending to infinity, e.g., at logarithmic rates, in order to cover a bigger set of parameters, eventually. The result below would then go through provided the constant $L$ in the credible sets $C_n(L)$ would also tend to infinity at a related rate.\] The condition requires that the contributions of the blocks $({\theta}_N,\ldots,{\theta}_{N\rho})$ of coordinates to the $\ell^2$-norm of ${\theta}$ cannot surge over the contributions of earlier blocks as $N\rightarrow\infty$. Sequences ${\theta}$ of exact polynomial order ${\theta}_{i}^2\asymp i^{-1-2{\beta}}$ on the "boundary" of hyperrectangles are obvious examples, but so are the sequences ${\theta}_i\asymp i^qe^{-\zeta i^c}$ and the sequences ${\theta}_{i}^2\asymp(\log i)^q i^{-1-2{\beta}}$ (for $q\in{\mathbb{R}}$, $\zeta,c>0$). Furthermore, because the condition is not on the individual coordinates ${\theta}_i$, but on blocks of coordinates of increasing size, the set of polished tail sequences is in fact much larger than these coordinatewise regular examples suggest. In particular, the set includes the "self-similar" sequences. These were defined by @PicTri and employed by @GineNickl and @Bull, in the context of wavelet bases and uniform norms. An $\ell^2$ definition in the same spirit with reference to hyperrectangles is as follows. A parameter ${\theta}\in\Theta^{{\beta}}(M)$ is *self-similar* if, for some fixed positive constants ${\varepsilon}$, $N_0$ and $\rho\ge2$, $$\sum_{i=N}^{\rho N}{\theta}_{i}^2 \geq{\varepsilon}M N^{-2{\beta}}\qquad\forall N\ge N_0. \label{propSelfSim}$$ We denote the class of self-similar elements of $\Theta^{\beta}(M)$ by $\Theta_{ss}^{\beta}(M,{\varepsilon})$. The parameters $N_0$ and $\rho$ are fixed and omitted from the notation. If we think of ${\theta}$ as a sequence of Fourier coefficients, then the right-hand side of (\[propSelfSim\]) without ${\varepsilon}$ is the maximal energy at frequency $N$ of a sequence in a hyperrectangle of radius $\sqrt M$. Thus, (\[propSelfSim\]) requires that the total energy in every *block* of consecutive frequency components is a fraction of the energy of a typical signal: the signal looks similar at all frequency levels. Here the blocks increase with frequency (with lengths proportional to frequency), whence the required similarity is only on average over large blocks. Self-similar sequences are clearly polished tail sequences, with $L_0={\varepsilon}^{-1}$. Whereas the first refer to a particular regularity class, the latter do not. As polished tail sequences are defined by self-referencing, they might perhaps be considered "self-similar" in a generalized sense. We show in Theorem \[thmCoverage\] that the polished tail condition is sufficient for coverage by the credible sets (\[EqCredibleSet\]). Self-similarity is restrictive. For instance, the polished tail sequence ${\theta}_i=i^{-{\beta}-1/2}(\log i)^{-q/2}$ is contained in the hyperrectangle $\Theta^{\beta}(1)$ for every $q\ge0$, and also in $S^{\beta}(M)$ for some $M$ if $q>1$, but it is not self-similar for any $q>0$. This could be remedied by introducing (many) different types of self-similar sequences, but the self-referencing of polished tail sequences seems much more elegant. An alternative to condition (\[propSelfSim\]) would be the slightly weaker $$\sum_{i=N}^{\infty}\theta_i^2 \geq{\varepsilon}M N^{-2\beta}\qquad\forall N\ge N_0.$$ This removes the parameter $\rho$, but, as $\theta$ is assumed to be contained in the hyperrectangle $\Theta^{\beta}(M)$, it can be seen that this seemingly relaxed condition implies (\[propSelfSim\]) with ${\varepsilon}$ replaced by ${\varepsilon}/2$ and $\rho$ sufficiently large that $\sum_{i>\rho N} i^{-1-2{\beta}}< {\varepsilon}/2 N^{-2{\beta}}$. One should ask how many parameters are not polished tail or self-similar. We give three arguments that there are only few: topological, minimax, and Bayesian. A topological comparison of the classes of self-similar and non self-similar functions obviously depends on the chosen topology. From the proof of Theorem \[LemmaCounterExample\] it is clear that the lack of coverage is due to the tail behavior of the non self-similar (or nonpolished-tail) truth in the statement of the theorem. Hence, by modifying the tail behavior of an arbitrary sequence ${\theta}$ we can get a non self-similar sequence with asymptotic coverage 0. Similarly, every element of $\Theta^{\beta}(M)$ can be made self-similar by modifying its tail. So we see that in the $\ell^2$-norm topology the difference in size between the two classes does not become apparent. Both the self-similar and the bad, non self-similar truths are dense in $\Theta^{\beta}(M)$. Following @GineNickl, one can also consider matters relative to the finer smoothness topology on $\Theta^{\beta}(M)$. Similar to their Proposition 4, it can then be shown that the set of non self-similar functions is nowhere dense, while the set of self-similar functions is open and dense. This suggests that self-similarity is the norm rather than the exception, as is also expressed by the term *generic* in the topological sense. The minimax argument for the neglibility of nonpolished-tail sequences is that restriction to polished tail (or self-similar) truths does not reduce the statistical difficulty of the problem. We show below (see Proposition \[LemmaMinimaxRate\]) that restriction to self-similarity changes only the constant in the minimax risk for hyperrectangles and not the order of magnitude or the dependence on the radius of the rectangle. Similarly, the minimax risk over Sobolev balls is reduced by at most a logarithmic factor by a restriction to polished tail sequences (see Proposition \[PropositionSobolevSize\]). A third type of reasoning is that polished tail sequences are *natural* once one has adapted the Bayesian setup with priors of the form (\[prior\]). The following proposition shows that almost every realization from such a prior is a polished tail sequence for some $N_0$. By making $N_0$ large enough we can make the set of polished tail sequences have arbitrarily large prior probability. This is true for any of the priors $\Pi_{\alpha}$ under consideration. Thus, if one believes one of these priors, then one accepts the polished tail condition. The result may be compared to Proposition 4 of @Hoff and Proposition 2.3 of @Bull. Recall that $\Theta_{pt}(L_0,N_0,\rho)$ is the set of ${\theta}\in\ell _2$ that satisfy (\[EqPolishedTail\]). For every ${\alpha}>0$ the prior $\Pi_{\alpha}$ in (\[prior\]) satisfies$\Pi_{\alpha}(\bigcup_{N_0}\Theta_{pt}(2/{\alpha}+1,N_0,2) )=1$. Let ${\theta}_1,{\theta}_2,\ldots$ be independent random variables with ${\theta}_i\sim N(0,\break i^{-1-2{\alpha}})$, and let $\Omega_N$ be the event $\{\sum_{i\ge N}{\theta}_i^2>(2/{\alpha}+1)\sum_{i=N}^{2N}{\theta}_i^2\}$. By the Borel–Cantelli lemma it suffices to show that $\sum_{N\in{\mathbb{N}}} \Pi_{\alpha}(\Omega_N)<\infty$. We have that $$\begin{aligned} {\mathrm{E}}\Biggl(\frac{2+{\alpha}}{{\alpha}}\sum_{i=N}^{2N} {\theta}_i^2-\sum_{i \ge N}{\theta}_i^2 \Biggr) &=&\frac{2+{\alpha}}{{\alpha}}\sum _{i=N}^{2N} \frac{1}{i^{1+2{\alpha}}}-\sum _{i\ge N}\frac{1}{i^{1+2{\alpha}}} \\ &\ge&\frac{2}{{\alpha}}\sum_{i=N}^{2N} \frac{1}{i^{1+2{\alpha}}}-\int_{2N}^{\infty}x^{-1-2{\alpha}} \,dx \\ &\ge&\bigl(2^{-2{\alpha}-1}/{\alpha}\bigr) N^{-2{\alpha}}.\end{aligned}$$ Therefore, by Markov's inequality, followed by the Marcinkiewitz–Zygmund and Hölder inequalities, for $q\ge2$ and $r>1$, $$\begin{aligned} \Pi_{\alpha}(\Omega_N) &\lesssim& N^{2{\alpha}q}{\mathrm{E}}\Biggl {\vert}\sum_{i=N}^{2N}\bigl( {\theta}_i^2-{\mathrm{E}}{\theta}_i^2\bigr) -\sum _{i>2N}\bigl({\theta}_i^2-{\mathrm{E}}{\theta}_i^2\bigr)\Biggr{\vert}^q \\ &\lesssim& N^{2{\alpha}q} {\mathrm{E}}\biggl(\sum_{i\ge N} \bigl({\theta}_i^2-{\mathrm{E}}{\theta}_i^2 \bigr)^2 \biggr)^{q/2} \\ &\lesssim& N^{2{\alpha}q} \sum_{i\ge N} {\mathrm{E}}\bigl( {\theta}_i^2-{\mathrm{E}}{\theta}_i^2 \bigr)^qi^{r(q/2-1)} \biggl(\sum_{i\ge N}i^{-r} \biggr)^{q/2-1}.\end{aligned}$$ Since ${\mathrm{E}}({\theta}_i^2-{\mathrm{E}}{\theta}_i^2)^q\asymp i^{-2q{\alpha}-q}$, for $-2q{\alpha}-q+r(q/2-1)<-1$ (e.g., $q>2$ and $r$ close to 1), the right-hand side is of the order $N^{2{\alpha}q} N^{-2q{\alpha}-q+r(q/2-1)+1} \times\break N^{-(r-1)(q/2-1)}=N^{-q/2}$. This is summable for $q>2$. The next theorem is the main result of the paper. It states that when the parameter is restricted to polished tail sequences, the empirical Bayes credible ball $\hat C_n(L)$ is an honest, frequentist confidence set, if $L$ is not too small. In Sections \[SectionHyperrectangle\]–\[SectionSupersmooth\] this theorem will be complemented by additional results to show that $\hat C_n(L)$ has radius $r_{n,\gamma}(\hat{{\alpha}}_n)$ of minimax order over a range of regularity classes. Recall that $\Theta_{pt}(L_0)$ is the set of all polished tail sequences ${\theta}\in\ell^2$ for the given constant $L_0$, and $\Aupper$ is the constant in (\[EqDefinitionHata\]). \[thmCoverage\] For any $\Aupper, L_0, N_0$ there exists a constant $L$ such that $$\inf_{{\theta}_0\in\Theta_{pt}(L_0)} P_{{\theta}_0} \bigl({\theta}_0\in\hat C_n(L) \bigr)\rightarrow1.\label{eqPostCov}$$ Furthermore, for $\Aupper=\Aupper_n\leq\sqrt{\log n}/(4\sqrt{\log \rho\vee e})$ this is true with a slowly varying sequence \[$L:=L_n\lesssim(3\rho^{3(1+2p)})^{\Aupper _n}$ works\]. See Section \[secMain\]. The theorem shows that the sets $\hat C_n(L)$ are large enough to catch any truth that satisfies the polished tail condition, in the sense of *honest* confidence sets. With the choice $L$ as in the theorem, their coverage in fact tends to 1, so that they are *conservative* confidence sets. In @Bartek it was seen that for deterministic choices of ${\alpha}$, the constant $L$ cannot be adjusted so that exact coverage $1-\g$ results; and $L$ in (\[eqPostCov\]) may have to be larger than 1. Thus, the credible sets $\hat C_n(L)$ are not exact confidence sets, but in combination with the results of Sections \[SectionHyperrectangle\]–\[SectionSupersmooth\] the theorem does indicate that their order of magnitude is correct in terms of frequentist confidence statements. As the credible sets result from a natural Bayesian procedure, this message is of interest by itself. In the next subsections we complement this by showing that the good coverage is not obtained by making the sets $\hat C_n(L)$ unduly large. On the contrary, their radius $Lr_{n,\g}(\hat{\alpha}_n)$ is of the minimax estimation rate for various types of models. In fact, it is immediate from the definition of the radius in (\[eqradius\]) that $r_{n,\g}(\hat{\alpha}_n)=O_P({\varepsilon}_n)$, whenever the posterior distribution contracts to the true parameter at the rate ${\varepsilon}_n$, in the sense that for every $M_n\rightarrow\infty$, $$E_{{\theta}_0} \Pi_{\hat{\alpha}_n} \bigl({\theta}\dvtx {\Vert}{\theta}-{\theta}_0 {\Vert}> M_n{\varepsilon}_n\mid X \bigr) \rightarrow0.$$ Such contraction was shown in @KSzVZ to take place at the (near) minimax rate ${\varepsilon}_n=n^{-{\beta}/(2{\beta}+2p+1)}$ uniformly in parameters ranging over Sobolev balls $S^{\beta}(M)$, adaptively in the regularity level ${\beta}$. In the next subsections we refine this in various ways: we also consider other models, and give refined and oracle statements for the behavior of the radius under polished tail or self-similar sequences. Generally speaking, the size of the credible sets $\hat C_n(L)$ are of (near) optimal size whenever the empirical Bayes posterior distribution (\[EqEBPosteriorDistribution\]) contracts at the (near) optimal rate. This is true for many but not all possible models for two reasons. On the one hand, the choice of priors with variances $i^{-1-2{\alpha}}$, for some ${\alpha}$, is linked to a certain type of regularity in the parameter. These priors yield a particular collection of posterior distributions $\Pi_{\alpha}(\cdot\mid X)$, and even the best possible (or oracle) choice of the tuning parameter ${\alpha}$ procedure is restricted to work through this collection of posterior distributions. Thus, the resulting procedure cannot be expected to be optimal for every model. One may think, for instance, of a model defined through a wavelet expansion, which has a double index and may not fit the Sobolev scale. Second, even in a situation that the collection $\Pi_{\alpha}(\cdot\mid X)$ contains an optimal candidate, the empirical Bayes procedure (\[EqDefinitionHata\]), linked to the likelihood, although minimax over the usual models, may fail to choose the optimal ${\alpha}$ for other models. Other empirical Bayes procedures sometimes perform better, for instance, by directly relating to the bias–variance trade-off. The radii $r_{n,\g}({\alpha})$ of the credible sets are decreasing in ${\alpha}$. Hence, if the empirical Bayes choice $\hat{\alpha}_n$ in (\[EqDefinitionHata\]) is restricted to a bounded interval $[0,\Aupper]$, then the credible set $\hat C_n(L)$ has radius not smaller than $r_{n,\g}(A)$, which is bigger than necessary if the true parameter has greater "regularity" than $A$. By the second statement of the theorem this can be remedied by choosing $A$ dependent on $n$, at the cost of increasing the radius by a slowly varying term. Hyperrectangles {#SectionHyperrectangle} --------------- The *hyperrectangle* $\Theta^{\beta}(M)$ of order ${\beta}$ and radius $M$ is defined in (\[defhyperrectangle\]). The minimax risk for this model in the case of the direct (not inverse) problem where $\kappa_i=1$ is given in @DonohoLiuMcGibbon. A slight variation of their proof gives that the minimax risk for square loss in our problem is bounded above and below by multiples of $M^{{(1+2p)}/{(1+2{\beta}+2p)}} n^{-{2{\beta}}/({1+2{\beta}+2p})}$, where the constant depends on $C$ and $p$ in (\[assumpinverse\]) only. Furthermore, this order does not change if the hyperrectangle is reduced to self-similar sequences. \[LemmaMinimaxRate\]\[LemmaComplexity\] Assume (\[assumpinverse\]). For all ${\beta}, M > 0$, $$\inf_{\hat{{\theta}}_n}\sup_{{\theta}_0\in\Theta^{\beta}(M)}E_{{\theta}_0} {\Vert}\hat{{\theta}}_n-{\theta}_0{\Vert}^2\asymp M^{\vafrac{1+2p}{1+2{\beta}+2p}} n^{-\afrac{2{\beta}}{1+2{\beta}+2p}},$$ where the infimum is over all estimators. This remains true if $\Theta^{\beta}(M)$ is replaced by $\Theta_{ss}^{\beta}(M,{\varepsilon})$, for any sufficiently small ${\varepsilon}>0$. The problem of estimating $({\theta}_i)$ based on the data (\[eqdata\]) is equivalent to estimating $({\theta}_i)$ based on independent $Y_1, Y_2,\ldots$ with $Y_i\sim N({\theta}_i,\break n^{-1}\kappa_i^{-2})$. As explained in @DonohoLiuMcGibbon (who consider identical variances instead of ${\sigma}_i^2=n^{-1}\kappa_i^{-2}$ depending on $i$, but this does not affect the ), the minimax estimator for a given hyperrectangle is the vector of estimators $T=(T_1,T_2,\ldots)$, where $T_i$ is the minimax estimator in the problem of estimating ${\theta}_i$ based on the single $Y_i\sim N({\theta}_i, {\sigma}_i^2)$, for each $i$, where it is known that ${\theta}_i^2\le M_i:=Mi^{-1-2{\beta}}$. Furthermore, @DonohoLiuMcGibbon show that in these univariate problems the minimax risk when restricting to estimators $T_i(Y_i)$ that are linear in $Y_i$ is at most $5/4$ times bigger than the (unrestricted, true) minimax risk, where the former *linear minimax risk* is easily computed to be equal to $M_i{\sigma}_i^2/(M_i+{\sigma}_i^2)$. Thus, the minimax risk in the present situation is up to a factor $5/4$ equal to $$\sum_{i=1}^\infty\frac{M_i{\sigma}_i^2}{M_i+{\sigma}_i^2} =\sum _{i=1}^\infty\frac{i^{-1-2{\beta}}Mn^{-1}\kappa_i^{-2}}{i^{-1-2{\beta}}M+n^{-1}\kappa_i^{-2}}.$$ Using assumption (\[assumpinverse\]) and Lemma \[LemmaTechLem10\] (with $l=1$, $m=0$, $r=1+2{\beta}+2p$, $s=2p$, and $Mn$ instead of $n$), we can evaluate this as the right-hand side of the proposition. To prove the final assertion, we note that the self-similar functions $\Theta_{ss}^{\beta}(M,{\varepsilon})$ are sandwiched between $\Theta^{\beta}(M)$ and the set $$\bigl\{{\theta}\dvtx \sqrt{{\varepsilon}M\rho^{1+2{\beta}}} i^{-1/2-{\beta}} \leq{\theta}_i\leq\sqrt{M} i^{-1/2-{\beta}}\mbox{ for all }i \bigr\}.$$ Likewise, the minimax risk for $\Theta_{ss}^{\beta}(M,{\varepsilon})$ is sandwiched between the minimax risks of these models. The smaller model, given in the display, is actually also a hyperrectangle, which can be shifted to the centered hyperrectangle $\Theta^{{\beta}}((1-\sqrt{{\varepsilon}\rho^{1+2{\beta}}})^2M/4)$. By shifting the observations likewise we obtain an equivalent experiment. The $\ell^2$-loss is equivariant under this shift. Therefore, the minimax risks of the smaller and bigger models in the sandwich are proportional to $(1-\sqrt{{\varepsilon}\rho^{1+2{\beta}}})^2 M^{({1+2p})/({1+2{\beta}+2p})} n^{-2{\beta}/(1+2{\beta}+2p)}/4$ and $M^{({1+2p})/({1+2{\beta}+2p})} n^{-2{\beta}/(1+2{\beta}+2p)}$, respectively, by the preceding paragraph. Theorem \[thmCoverage\] shows that the credible sets $C_n(L)$ cover self-similar parameters in $\Theta^{\beta}(M)$ and, more generally, parameters satisfying the polished tail condition, uniformly in regularity parameters ${\beta}\in[0,\Aupper]$ and also uniformly in the radius $M$ (but dependent on ${\varepsilon}$ in the definition of self-similarity or $L_0$ in the definition of the polished tail condition). Straightforward adaptation of the proof of Theorem 2 in @KSzVZ shows that the empirical Bayes posterior distribution $\Pi_{\hat{\alpha}_n}(\cdot\mid X)$ contracts to the true parameter at the minimax rate, by a logarithmic factor, uniformly over any hyperrectangle $\Theta^{\beta}(M)$ with ${\beta}\le\Aupper$. This immediately implies that the radius $r_{n,\g }(\hat{\alpha}_n)$ is at most a logarithmic factor larger than the minimax rate, and hence the size of the credible sets $\hat C_n(L)$ adapts to the scale of hyperrectangles. Closer inspection shows that the logarithmic factor in this result arises from the bias term and is unnecessary for the radius $r_{n,\g}(\hat{\alpha}_n)$. Furthermore, this radius also adapts to the constant $M$ in the optimal manner. \[lemmaPropositionRectangleMinimxRateEB\] For every ${\beta}\in(0,\Aupper]$ and $M > 0$, $$\inf_{{\theta}_0 \in\Theta^{\beta}(M)} P_{{\theta}_0} \bigl( r_{n,\gamma}(\hat {\alpha}_n)\le K M^{\vafrac{1/2+p}{1+2{\beta}+2p}}n^{-\afrac{{\beta}}{1+2{\beta}+2p}} \bigr) \rightarrow1. $$ The proof of this proposition and of all further results in this section is given in Section \[SectionProofHyper\]. This encouraging result can be refined to an oracle type result for self-similar true parameters. Recall that $\Theta_{ss}^{\beta}(M,{\varepsilon})$ is the collection of self-similar parameters in $\Theta^{\beta}(M)$. \[TheoremRectangleRadiusSelfSimilar\] For all ${\varepsilon}$ there exists a constant $K({\varepsilon})$ such that, for all $M$, $$\inf_{{\theta}_0\in\Theta_{ss}^{\beta}(M,{\varepsilon})}P_{{\theta}_0} \Bigl(r_{n,\gamma }^2( \hat{{\alpha}}_n) \leq K({\varepsilon}) \inf_{{\alpha}\in[0,\Aupper]} E_{{\theta}_0}{\Vert}\hat{\theta}_{n,{\alpha}}-{\theta}_0{\Vert}^2 \Bigr)\rightarrow1. \label{eqadaptivity}$$ The theorem shows that for self-similar truths ${\theta}_0$ the square radius of the credible set is bounded by a multiple of the mean square error$\inf_{{\alpha}\in[0,\Aupper]} E_{{\theta}_0}{\Vert}\hat{\theta}_{n,{\alpha}}-{\theta}_0{\Vert}^2$ of the best estimator in the class of all Bayes estimators of the form $\hat{\theta}_{n,{\alpha}}$, for ${\alpha}\in [0,\Aupper]$. The choice ${\alpha}$ can be regarded as made by an oracle with knowledge of ${\theta}_0$. The class of estimators $\hat{\theta}_{n,{\alpha}}$ is not complete, but rich. In particular, the proposition below shows that it contains a minimax estimator for every hyperrectangle $\Theta^{\beta}(M)$ with ${\beta}\le\Aupper$. \[PropositionRectangleCompletenessBayesEstimators\] For every ${\beta}\in(0,\Aupper]$ and $M > 0$, $$\begin{aligned} \inf_{{\alpha}\in[0,\Aupper]} \sup_{{\theta}_0\in\Theta^{{\beta}}(M)}E_{{\theta}_0} {\Vert}\hat{\theta}_{n,{\alpha}}-{\theta}_0{\Vert}^2 &\lesssim& M^{\vafrac{1+2p}{1+2p+2{\beta}}}n^{-\afrac{2{\beta}}{1+2{\beta}+2p}}.\end{aligned}$$ Within the scale of hyperrectangles it is also possible to study the asymptotic behavior of the empirical Bayes regularity $\hat{\alpha}_n$ and corresponding posterior distribution. For parameters in $\Theta_{ss}^{\beta}(M)$ the empirical Bayes estimator estimates $\beta$, which might thus be considered a "true" regularity ${\beta}$. \[lemmaBoundsForAinHyperrectangle\] For any $0 < {\beta}\le\Aupper-1$ and $M \ge1$, there exist constants $K_1$ and $K_2$ such that $\Pr_{{\theta}_0}({\beta}-K_1/\log n\le\hat{\alpha}_n\le{\beta}+K_2/\log n)\rightarrow1$ uniformly in ${\theta}_0\in\Theta_{ss}^{\beta}(M,{\varepsilon})$. Inspection of the proof of the lemma shows that the constant $K_2$ will be negative for large enough $M$, meaning that the empirical Bayes estimate $\hat {\alpha}_n$ will then actually slightly (up to a $1/\log n$ factor) undersmooth the "true" regularity. The assumption that ${\theta}_0\in\Theta_{ss}^{\beta}(M,{\varepsilon})$ implies not only that ${\theta}_0$ is of regularity ${\beta}$, but also that within the rectangle it is at a distance proportional to $M$ from the origin. Thus, an increased distance from the origin in a rectangle of fixed regularity ${\beta}$ is viewed by the empirical Bayes procedure as "a little less smooth than ${\beta}$." (This is intuitively reasonable and perhaps the smoothness of such ${\theta}_0$ should indeed be viewed as smaller than $\beta$.) The "undersmoothing" is clever, and actually essential for the coverage of the empirical Bayes credible sets, uniformly over all radii $M>0$. Sobolev balls {#SectionSobolevballs} ------------- The *Sobolev ball* $S^{\beta}(M)$ of order ${\beta}$ and radius $M$ is defined in (\[defsobolevspace\]). The minimax risk for this model is well known to be of the same order as for the hyperrectangles considered in Section \[SectionHyperrectangle\] \[see @CavURE or @Cavalier2\]. A restriction to polished tail sequences decreases this by at most a logarithmic factor. \[PropositionSobolevSize\] Assume (\[assumpinverse\]). For all ${\beta}, M > 0$, $$\begin{aligned} && \inf_{\hat{{\theta}}_n}\sup_{{\theta}_0\in S^{\beta}(M)\cap\Theta _{pt}(L_0)}E_{{\theta}_0} {\Vert}\hat{{\theta}}_n-{\theta}_0{\Vert}^2 \\ &&\qquad \gtrsim M^{\vafrac{1+2p}{1+2{\beta}+2p}} n^{-\afrac{2{\beta}}{1+2{\beta}+2p}} (\log n)^{-\vafrac {2+4p}{1+2{\beta}+2p}},\end{aligned}$$ where the infimum is over all estimators. The set $S^{\beta}(M)\cap\Theta_{pt}(L_0)$ contains the set $$\bigl\{{\theta}\in\ell_2\dvtx {\varepsilon}M i^{-1-2{\beta}}(\log i)^{-2}\le{\theta}_i\le{\varepsilon}' M i^{-1-2{\beta}}(\log i)^{-2}\bigr\},$$ for suitable ${\varepsilon}<{\varepsilon}'$. This is a translate of a hyperrectangle delimited by the rate sequence $Mi^{-1-2{\beta}}(\log i)^{-2}$. The minimax risk over this set can be computed as in the proof of Proposition \[LemmaMinimaxRate\]. By Theorem 2 of @KSzVZ the empirical Bayes posterior distribution $\Pi_{\hat{\alpha}_n}(\cdot\mid X)$ (with $\Aupper$ set to $\log n$) contracts to the true parameter at the minimax rate, up to a logarithmic factor, uniformly over any Sobolev ball $S^{\beta}(M)$. This implies that the radius $r_{n,\g}(\hat{\alpha}_n)$ is at most a logarithmic factor larger than the minimax rate, and hence the size of the credible sets $\hat C_n(L)$ adapts to the Sobolev scale. Again, closer inspection shows that the logarithmic factors do not enter into the radii, and the radii adapt to $M$ in the correct manner. The proof of the following theorem can be found in Section \[SectionProofSobolev\]. \[TheoremSobolevRadiusPolishedTail\] There exists a constant $K$ such that, for all ${\beta}\in[0,\Aupper]$, $M$ and $L_0$, $$\inf_{{\theta}_0\in S^{\beta}(M)}P_{{\theta}_0} \bigl(r_{n,\gamma}^2( \hat{{\alpha}}_n) \leq K M^{\vafrac{1+2p}{1+2{\beta}+2p}} n^{-\afrac{2{\beta}}{1+2{\beta}+2p}} \bigr) \rightarrow1.$$ Thus, the empirical Bayes credible sets are honest confidence sets, uniformly over the scale of Sobolev balls (for ${\beta}\in[0,\Aupper]$ and $M>0$) intersected with the polished tail sequences, of the minimax size over every Sobolev ball. If $\Aupper$ in (\[EqDefinitionHata\]) is allowed to grow at the order $\sqrt{\log n}$, then these results are true up to logarithmic factors. Supersmooth parameters {#SectionSupersmooth} ---------------------- Classes of *supersmooth parameters* are defined by, for fixed $N, M, c, d>0$, $$\begin{aligned} C^{00}(N_0,M)&=&\Bigl\{{\theta}\in\ell^2\dvtx \sup _{i> N_0}{\vert}{\theta}_i{\vert}=0\mbox{ and } \sup_{i\le N_0}{\vert}{\theta}_i{\vert}\leq M^{1/2}\Bigr\},\label{defcnull} \\ S^{\infty,c,d}(M)&=&\Biggl\{{\theta}\in\ell^2\dvtx \sum _{i=1}^\infty e^{ci^d}{\theta}_i^2 \le M\Biggr\}.\label{defsinfty}\end{aligned}$$ The minimax rate over these classes is $n^{-1/2}$ or $n^{-1/2}$ up to a logarithmic factor. Every class $C_{00}(N_0,M)$ for fixed $N_0$ and arbitrary $M$ consists of polished tail sequences only. If the empirical Bayes regularity $\hat{\alpha}_n$ in (\[EqDefinitionHata\]) is restricted to a compact interval $[0,\Aupper]$, then it will tend to the upper limit $\Aupper$ whenever the true parameter is in one of these supersmooth models. Furthermore, the bias of the posterior mean will be of order $O(1/n)$, which is negligible relative to its variance at ${\alpha}=\Aupper$ (which is the smallest over $[0,\Aupper]$). Coverage by the credible sets then results. More interestingly, for $\hat{\alpha}_n$ in (\[EqDefinitionHata\]) restricted to $[0,\log n]$, it is shown in @KSzVZ that the posterior distribution contracts at the rate $n^{-1/2}$ up to a lower order factor, for ${\theta}_0\in S^{\infty,c,1}$. Thus, the empirical Bayes credible sets then adapt to the minimal size over these spaces. Coverage is not automatic, but does take place uniformly in polished tail sequences by Theorem \[thmCoverage\]. The following theorem extends the findings on the sizes of the credible sets. \[TheoremSupersmoothRadiusPolishedTail\] There exists a constant $K$ such that, for $\Aupper=\Aupper_n=\sqrt {\log n}/(4\log\rho)$, $$\begin{aligned} \inf_{{\theta}_0\in C^{00}(N_0,M)}P_{{\theta}_0} \bigl(r_{n,\gamma}^2( \hat{{\alpha}}_n) \leq K e^{(3/2+3p)\sqrt{\log N_0}\sqrt{\log n}} n^{-1} \bigr) \rightarrow1,\label{rateC00} \\ \inf_{{\theta}_0\in S^{\infty,c,d}(M)}P_{{\theta}_0} \bigl(r_{n,\gamma }^2( \hat{{\alpha}}_n) \leq e^{(1/2+p)\sqrt{\log n}\log\log n} n^{-1} \bigr )\rightarrow 1.\label{rateSinfty}\end{aligned}$$ Simulation example {#secsim} ================== We investigate the uncertainty quantification of the empirical Bayes credible sets in an example. Assume that we observe the process $$\begin{aligned} X_t&=&\int_0^t\int _0^s {\theta}_0(u) \,du \,ds+ \frac{1}{\sqrt{n}}B_t, \qquad t \in[0,1],\end{aligned}$$ where $B$ denotes a standard Brownian motion and ${\theta}_0$ is the unknown function of interest. It is well known and easily derived that this problem is equivalent to (\[eqdata\]), with ${\theta}_{0,i}$ the Fourier coefficients of ${\theta}_0$ relative to the eigenfunctions $e_i(t) = \sqrt{2}\cos(\pi(i-1/2)t)$ of the Volterra integral operator $K{\theta}(t)=\int_0^{t}{\theta}(u)\,du$, and ${\kappa}_i=1/(i-1/2)/\pi$ the corresponding eigenvalues. In particular, $p=1$ in (\[assumpinverse\]), that is, the problem is mildly ill posed. For various signal-to-noise ratios $n$ we simulate data from this model corresponding to the true function $${\theta}_0(t)=\sum_{i=1}^{\infty} \bigl(i^{-3/2}\sin(i)\bigr)\sqrt{2}\cos\bigl(\pi(i-1/2)t \bigr).\label{exselfsim}$$ This function ${\theta}_0$ is self-similar with regularity parameter ${\beta}=1$. ![Empirical Bayes credible sets. The true function is drawn in black, the posterior mean in blue and the credible set in grey. We have $n=10^3,10^6,10^8$ and $10^{10}$, respectively.<span data-label="figure1"></span>](1270f01.eps) In Figure \[figure1\] we visualize $95\%$ credible sets for ${\theta}_0$ (gray), the posterior mean (blue) and the true function (black), by simulating $2000$ draws from the empirical Bayes posterior distribution and plotting the $95\%$ draws out of the 2000 that are closest to the posterior mean in the $L^2$-sense. The credible sets are drawn for $n = 10^4, 10^6, 10^8$ and $10^{10}$, respectively. The pictures show good coverage, as predicted by Theorem \[thmCoverage\]. We note that we did not blow up the credible sets by a factor $L > 1$. To illustrate the negative result of Theorem \[LemmaCounterExample\], we also computed credible sets for a "bad truth." We simulated data using the following function: $$\begin{aligned} &&{\theta}_0(t)=\sum_{i=1}^{\infty} {\theta}_{0,i}\sqrt{2}\cos\bigl(\pi(i-1/2)t\bigr),\end{aligned}$$ where $$\begin{aligned} {\theta}_{0,i}=\cases{ 8, &\quad for $i=1$, \vspace*{3pt}\cr 2, &\quad for $i=3$, \vspace*{3pt}\cr -2, &\quad if $i=50$, \vspace*{3pt}\cr i^{-3/2}, &\quad if $2^{4^j}<i \leq2*2^{4^j}$, for $j\geq3$, \vspace*{3pt}\cr 0, &\quad else.}\end{aligned}$$ Figure \[figureCounterExample\] shows the results, with again the true function ${\theta}_0$ in black, the posterior mean in blue and the credible sets in gray. The noise levels are $n=20,50,10^3,6*10^4,5*10^5$ and $5*10^6$, respectively. As predicted by Theorem \[LemmaCounterExample\], the coverage is very bad along a subsequence. For certain values of $n$ the posterior mean is far from the truth, yet the credible sets very narrow, suggesting large confidence in the estimator. ![Empirical Bayes credible sets for a non self-similar function. The true function is drawn in black, the posterior mean in blue and the credible sets in grey. From left to right and top to bottom we have $n=20,50,10^3,6*10^4,5*10^5$ and $5*10^6$.<span data-label="figureCounterExample"></span>](1270f02.eps) Proof of Theorem 3.6 {#secproofcov} ==================== \[secMain\] The proof of Theorem \[thmCoverage\] is based on a characterization of two deterministic bounds on the data-driven choice $\hat{\alpha}_n$ of the smoothing parameter. Following @KSzVZ and @Szabo, define a function $h_n=h_n(\cdot; {\theta}_0)$ by $$h_n({\alpha};{\theta}_0)= \frac{1+2{\alpha}+2p}{n^{1/(1+2{\alpha}+2p)}\log n} \sum _{i=1}^{\infty}\frac{n^2i^{1+2{\alpha}}(\log i){\theta}_{0,i}^2}{ (i^{1+2{\alpha}+2p}+n)^2}\label{eqhnalpha}, \qquad{\alpha}\ge0.$$ Next define $$\begin{aligned} \underline{{\alpha}}_n({\theta}_0) &=&\inf\bigl\{{\alpha}\in[0, \Aupper]\dvtx h_n({\alpha};{\theta}_0)\geq1/\bigl(16C^8 \bigr)\bigr\},\label {defLowAlpha} \\ \overline{{\alpha}}_n({\theta}_0) &=&\sup\bigl\{{\alpha}\in[0, \Aupper]\dvtx h_n({\alpha};{\theta}_0)\leq8C^8 \bigr \}\label{defUppAlpha}.\end{aligned}$$ An infimum or supremum over an empty set can be understood to be $\Aupper$ or 0, respectively. The value $\Aupper$ is as in (\[EqDefinitionHata\]). If it depends on $n$, then this dependence is copied into the definitions. The following theorem shows that uniformly in parameters ${\theta}_0$ that satisfy the polished tail condition, the sequences $\underline{\alpha}_n({\theta}_0)$ and $\overline {\alpha}_n({\theta}_0)$ capture $\hat{\alpha}_n$ with probability tending to one. Furthermore, these sequences are at most a universal multiple of $1/\log n$ (or slightly more if $\Aupper$ depends on $n$) apart, leading to the same "rate" $n^{-{\alpha}/(1+2{\alpha}+2p)}$, again uniformly in polished tail sequences ${\theta}_0$. \[thmmle\] For every $L_0\ge1$, $$\begin{aligned} \inf_{{\theta}_0\in\Theta_{pt}(L_0)} P_{{\theta}_0} \bigl(\underline {\alpha}_n({\theta}_0)\le\hat{\alpha}_n\le\overline{\alpha}_n({\theta}_0) \bigr)&\rightarrow& 1,\label{eqm1} \\ \sup_{{\theta}_0\in\Theta_{pt}(L_0)}\frac{n^{-2\underline{\alpha}_n({\theta}_0)/(1+2\underline{\alpha}_n({\theta}_0)+2p)}}{ n^{-2\overline{\alpha}_n({\theta}_0)/(1+2\overline{\alpha}_n({\theta}_0)+2p)}}&\le& K_{3,n},\label{eqDiffBounds}\end{aligned}$$ for $\log n\ge2+4\Aupper+2p\vee C_0$, where $C_0$ depends on $N_0$, $$\begin{aligned} && K_{3,n}\le c \bigl(2^9C^{16}L_0^2 \rho^{5+10\Aupper+6p}\bigr) ^{1+2p},\end{aligned}$$ for some universal constant $c$. The proof of (\[eqm1\]) is similar to proofs given in @KSzVZ. However, because the exploitation of the polished tail condition and the required uniformity in the parameters are new, we provide a complete proof of this assertion in the . Here we only prove inequality (\[eqDiffBounds\]). Let $N_{\alpha}=n^{1/(1+2{\alpha}+2p)}$ and set $$\Theta_j=\sum_{i=\rho^{j-1}+1}^{\rho^j} {\theta}_i^2.$$ Then the polished tail condition (\[EqPolishedTail\]) implies that $\Theta_j\le L_0 \Theta_{j'}$ for all $j\ge j'$, whence $$\begin{aligned} h_n({\alpha};{\theta}) &=& \frac{1}{N_{\alpha}\log N_{\alpha}}\sum _{i=1}^\infty\frac{n^2i^{1+2{\alpha}}{\theta}_i^2\log i}{(i^{1+2{\alpha}+2p}+n)^2} \\ &\le&\frac{1}{N_{\alpha}\log N_{\alpha}}\sum_{j=1}^\infty \Theta_j\frac {n^2\rho^{j(1+2{\alpha})}j\log\rho}{ (\rho^{(j-1)(1+2{\alpha}+2p)}+n)^2}.\end{aligned}$$ We get a lower bound if we swap the $j$ and $j-1$ between numerator and denominator. \[The term $(j-1)\log\rho$ that then results in the numerator is a nuisance for $j=1$; instead of $(j-1)\log\rho= \log\rho^{j-1}$, we may use $\log2$ if $j=1$, as the term $i=1$ does not contribute.\] Define $J_{\alpha}$ to be an integer such that $$\begin{aligned} J_{\alpha}= \biggl\lfloor\frac{\log n}{(\log\rho)(1+2{\alpha}+2p)} \biggr\rfloor\nonumber \\ \eqntext{\mbox{hence } \rho^{J_{\alpha}(1+2{\alpha}+2p)}\leq n< \rho^{1+2{\alpha}+2p}\rho^{J_{\alpha}(1+2{\alpha}+2p)}.}\end{aligned}$$ Under the polished tail condition, $$\begin{aligned} && \sum_{j>J_{\alpha}}\Theta_j\frac{n^2\rho^{j(1+2{\alpha})}j\log\rho} {(\rho ^{(j-1)(1+2{\alpha}+2p)}+n)^2} \\ &&\qquad \le L_0\Theta_{J_{\alpha}}\sum_{j>J_{\alpha}}n^2 \rho^{-j(1+2{\alpha}+4p)}j(\log\rho)\rho^{2(1+2{\alpha}+2p)} \\ &&\qquad \lesssim L_0\Theta_{J_{\alpha}}n^2 \rho^{-J_{\alpha}(1+2{\alpha}+4p)}J_{\alpha}(\log\rho) \rho^{2(1+2{\alpha}+2p)} \\ &&\qquad \leq L_0\Theta_{J_{\alpha}}nJ_{\alpha}(\log\rho) \rho^{3(1+2{\alpha}+2p)-J_{{\alpha}}2p}.\end{aligned}$$ The constant in $\lesssim$ is universal. \[We have $\sum_{j>J} jx^{j}=x^{J+1}(J+1) ((1-x)^{-1}-(1-x)^{-2}/(J+1) )$ for $0<x<1$, and $x=\rho^{-(1+2{\alpha}+4p)}\le\rho^{-1}\le1/2$.\] The $J_{\alpha}$th term of the series on the left-hand side is $$\Theta_{J_{\alpha}}\frac{n^2\rho^{J_{\alpha}(1+2{\alpha})}J_{\alpha}\log\rho} {(\rho ^{(J_{\alpha}-1)(1+2{\alpha}+2p)}+n)^2} \geq(1/4) \Theta_{J_{\alpha}} n J_{\alpha}\rho^{-J_{{\alpha}}2p}\log\rho.$$ Up to a factor $L_0\rho^{3(1+2{\alpha}+2p)}$ this has the same order of magnitude as the right-hand side of the preceding display, whence $$h_n({\alpha};{\theta}) \lesssim\bigl(1+L_0\rho^{3+6{\alpha}+6p} \bigr) \frac{1}{N_{\alpha}\log N_{\alpha}} \sum_{j=1}^{J_{\alpha}} \Theta_j\frac{n^2\rho^{j(1+2{\alpha})}j(\log\rho)}{ (\rho^{(j-1)(1+2{\alpha}+2p)}+n)^2}.$$ Because $\rho^{j(1+2{\alpha}+2p)}+n\le2n$ for $j\le J_{\alpha}$, we also have $$\begin{aligned} h_n({\alpha};{\theta}) &\ge&\frac{1}{N_{\alpha}\log N_{\alpha}}\sum _{j=1}^{J_{\alpha}}\frac{\Theta_j\rho ^{(j-1)(1+2{\alpha})}\log(\rho^{j-1}\vee2)}{4} \nonumber \\ &\ge&\frac{1}{N_{\alpha}\log N_{\alpha}}\frac{\rho^{-(1+2{\alpha})}}8\sum_{j=1}^{J_{\alpha}} \Theta_j\rho^{j(1+2{\alpha})}j(\log\rho\wedge1).\end{aligned}$$ (Note that $\log2\approx0.69\ge1/2$ and $j-1\ge j/2$ for $j\ge2$.) Now fix ${\alpha}_1\le{\alpha}_2$. Then $J_{{\alpha}_2}\le J_{{\alpha}_1}$ and $\rho ^{1+2{\alpha}_1}\le\rho^{1+2{\alpha}_2}$ and $$\begin{aligned} && \frac{h_n({\alpha}_1;{\theta})}{h_n({\alpha}_2;{\theta})} \\ &&\qquad \lesssim \frac{N_{{\alpha}_2}\log N_{{\alpha}_2}}{N_{{\alpha}_1}\log N_{{\alpha}_1}} \frac {(1+L_0\rho^{3+6{\alpha}_1+6p}) (\sum_{j=1}^{J_{{\alpha}_2}}+\sum_{j=J_{{\alpha}_2}}^{J_{{\alpha}_1}})\Theta_j\rho^{j(1+2{\alpha}_1)}j(\log\rho)}{ \rho^{-(1+2{\alpha}_2)}\sum_{j=1}^{J_{{\alpha}_2}}\Theta_j\rho^{j(1+2{\alpha}_2)}j(\log\rho\wedge1)} \\ &&\qquad \lesssim \frac{N_{{\alpha}_2}\log N_{{\alpha}_2}}{N_{{\alpha}_1}\log N_{{\alpha}_1}}\bigl(1+L_0\rho^{3+6{\alpha}_1+6p}\bigr) \rho^{1+2{\alpha}_2} \biggl(1+\frac{L_0\Theta_{J_{{\alpha}_2}}\sum_{j=J_{{\alpha}_2}}^{J_{{\alpha}_1}}\rho^{j(1+2{\alpha}_1)}j}{ \Theta_{J_{{\alpha}_2}}\rho^{J_{{\alpha}_2}(1+2{\alpha}_2)}J_{{\alpha}_2}} \biggr) \\ &&\qquad \lesssim\frac{N_{{\alpha}_2}\log N_{{\alpha}_2}}{N_{{\alpha}_1}\log N_{{\alpha}_1}}\bigl(1+L_0\rho^{3+6{\alpha}_1+6p}\bigr) \rho^{1+2{\alpha}_2} \biggl(1+L_0\frac{\rho^{J_{{\alpha}_1}(1+2{\alpha}_1)}J_{{\alpha}_1}}{ \rho^{J_{{\alpha}_2}(1+2{\alpha}_2)}J_{{\alpha}_2}} \biggr) \\ &&\qquad \leq\frac{N_{{\alpha}_2}\log N_{{\alpha}_2}}{N_{{\alpha}_1}\log N_{{\alpha}_1}}\bigl(1+L_0\rho^{3+6{\alpha}_1+6p}\bigr) \rho^{2+4{\alpha}_2} \biggl(1+L_0\frac{J_{{\alpha}_1}}{J_{{\alpha}_2}} \biggr) \\ &&\qquad \leq\frac{N_{{\alpha}_2}}{N_{{\alpha}_1}}\frac{1+2{\alpha}_1+2p}{1+2{\alpha}_2+2p}\bigl(1+L_0 \rho^{3+6{\alpha}_1+6p}\bigr) \rho^{2+4{\alpha}_2} \biggl(1+2L_0 \frac{1+2{\alpha}_2+2p}{1+2{\alpha}_1+2p} \biggr).\end{aligned}$$ \[We use $\sum_{j=I}^J x^j=x^I(x^{J-I+1}-1)/(x-1)\le x^J x/(x-1)\lesssim x^J$, for $x=\rho^{1+2{\alpha}}\ge\rho$.\] Since $\underline{\alpha}_n\le\overline{\alpha}_n$, there is nothing to prove in the trivial cases $\underline{\alpha}_n=\Aupper$ or $\overline{\alpha}_n=0$. In the other cases it follows that $h_n(\underline{\alpha}_n;{\theta}_0)\ge 1/(16C^8)$ and $h_n(\overline{\alpha}_n;{\theta}_0)\le8 C^8$. Then the left-hand side of the preceding display with ${\alpha}_1=\underline{\alpha}_n$ and ${\alpha}_2=\overline{\alpha}_n$ is bounded from below by $1/(128C^{16})$. After taking the $(1+2p)$th power of both sides and rearranging the inequality we get $$\begin{aligned} && \bigl(2^8C^{16}\bigr)^{1+2p}\bigl(1+L_0 \rho^{3+6\Aupper+6p}\bigr)^{1+2p}\rho^{(2+4\Aupper)(1+2p)}(1+2L_0)^{1+2p} \\ &&\qquad \gtrsim N_{\underline{\alpha}_n}^{1+2p}/N_{\overline{\alpha}_n}^{1+2p} =N_{\underline{\alpha}_n}^{-2\underline{\alpha}_n}/N_{\overline{\alpha}_n}^{-2\overline{\alpha}_n}.\end{aligned}$$ This concludes the proof of Theorem \[thmmle\]. We proceed to the proof of Theorem \[thmCoverage\]. Recall the definition of the posterior distribution $\Pi_{\alpha}(\cdot\mid X)$ in (\[PostDist\]). For notational convenience denote the mean of the posterior distribution (\[PostDist\]) by $\hat{\theta}_{\alpha}$ and the radius $r_{n,\gamma}({\alpha})$ defined by (\[eqradius\]) by $r({\alpha})$. Furthermore, let $$\label{EqDefWB} W({\alpha}) = \hat{\theta}_{\alpha}- E_{{\theta}_0}\hat {\theta}_{\alpha}\quad\mbox{and}\quad B({\alpha};{\theta}_0) = E_{{\theta}_0}\hat{\theta}_{\alpha}- {\theta}_0$$ be the centered posterior mean and the bias of the posterior mean, respectively. The radius $r({\alpha})$ and the distribution of the variable $W({\alpha})$ under ${\theta}_0$ are free of ${\theta}_0$. On the other hand, the bounds $\underline{\alpha}_n$ and $\overline{\alpha}_n$ do depend on ${\theta}_0$, but we shall omit this from the notation. Because the radius of the credible set is defined to be infinite if $\hat{\alpha}_n=0$, it is not a loss of generality to assume that $\overline{\alpha}_n>0$. For simplicity we take $\Aupper$ in (\[EqDefinitionHata\]) independent of $n$, but make the dependence of constants on $\Aupper$ explicit, so that the general case follows by inspection. We prove below that there exist positive parameters $ C_1, C_2, C_3$ that depend on $C, \Aupper, p, L_0, \rho$ only such that, for all ${\theta}_0 \in\Theta_{pt}(L_0)$, $$\begin{aligned} \label{r} \inf_{\underline{\alpha}_n \le{\alpha}\le\overline{\alpha}_n}r({\alpha}) &\ge& C_1 n^{-\afrac{\overline{\alpha}_n}{1+2\overline{\alpha}_n + 2p}}, \\ \label{b} \sup_{\underline{\alpha}_n \le{\alpha}\le\overline{\alpha}_n}\bigl{\Vert}B({\alpha}; {\theta}_0)\bigr{\Vert}&\le& C_2 n^{-\afrac{\overline{\alpha}_n}{1+2\overline{\alpha}_n + 2p}}, \\ \label{w} \inf_{{\theta}_0 \in\Theta_{pt}(L_0)}P_{{\theta}_0} \Bigl(\sup _{\underline {\alpha}_n \le{\alpha}\le\overline{\alpha}_n} \bigl{\Vert}W({\alpha})\bigr{\Vert}&\le& C_3n^{-\afrac{\overline{\alpha}_n}{1+2\overline{\alpha}_n + 2p}} \Bigr) \rightarrow1.\end{aligned}$$ We have ${\theta}_0\in\hat C_n(L)$ if and only if ${\Vert}\hat{\theta}_{\hat{\alpha}}-{\theta}_0{\Vert}\le L r(\hat{\alpha})$, which is implied by ${\Vert}W(\hat{\alpha}){\Vert}\le Lr(\hat{\alpha})-{\Vert}B(\hat{\alpha};{\theta}_0){\Vert}$, by the triangle inequality. Consequently, by (\[eqm1\]) of Theorem \[thmmle\], to prove (\[eqPostCov\]), it suffices to show that for $L$ large enough $$\inf_{{\theta}_0 \in\Theta_{pt}(L_0)}P_{{\theta}_0} \Bigl(\sup_{\underline {\alpha}_n \le{\alpha}\le\overline{\alpha}_n} \bigl{\Vert}W({\alpha})\bigr{\Vert}\le L \inf_{\underline{\alpha}_n \le{\alpha}\le \overline{\alpha}_n}r({\alpha}) - \sup _{\underline{\alpha}_n \le{\alpha}\le\overline{\alpha}_n}\bigl{\Vert}B({\alpha};{\theta}_0)\bigr {\Vert}\Bigr) \rightarrow1.$$ This results from the combination of (\[r\]), (\[b\]) and (\[w\]), for $L$ such that $C_3\le LC_1-C_2$. We are left to prove (\[r\]), (\[b\]) and (\[w\]). <span>Proof of Proof of (\[r\])</span> The radius $r({\alpha})$ is determined by the requirement that $P (U_n({\alpha})<r^2({\alpha}) )=1-\gamma$, for the random variable $U_n({\alpha})=\sum_i s_{i,n,{\alpha}}Z_i^2$, where $s_{i,n,{\alpha}}={\kappa}_i^{-2}/(i^{1+2{\alpha}}{\kappa}_i^{-2}+n)$ and $(Z_i)$ is an i.i.d. standard normal sequence. Because ${\alpha}\mapsto s_{i,n,{\alpha}}$ is decreasing in ${\alpha}$, the map ${\alpha}\mapsto r({\alpha})$ is nonincreasing, and hence the infimum in (\[r\]) is equal to $r(\overline{\alpha}_n)$. In view of (\[assumpinverse\]) and Lemma \[LemmaTechLem10\], the expected value and variance of $U_n({\alpha})$ satisfy, for $n\ge e^{1+2{\alpha}+2p}$, $$\begin{aligned} {\mathrm{E}}U_n({\alpha})&=&\sum_{i=1}^{\infty}s_{i,n,{\alpha}} \geq\frac{1}{C^{4}}\sum_{i=1}^{\infty} \frac{i^{2p}}{i^{1+2{\alpha}+2p}+n} \\ &\ge&\frac{1}{C^4(3^{1+2{\alpha}+2p}+1)} n^{-\afrac{2{\alpha}}{1+2{\alpha}+2p}}, \\ {\operatorname{var}}U_n({\alpha})&=&2\sum_{i=1}^{\infty}s_{i,n,{\alpha}}^2 \leq2C^{8}\sum_{i=1}^{\infty} \frac{i^{4p}}{(i^{1+2{\alpha}+2p}+n)^2} \\ &\le& 10 C^8n^{-\vafrac{1+4{\alpha}}{1+2{\alpha}+2p}}.\end{aligned}$$ We see that the standard deviation of $U_n({\alpha})$ is negligible compared to its expected value. This implies that all quantiles of $U_n({\alpha})$ are of the order ${\mathrm{E}}U_n({\alpha})$. More precisely, by Chebyshev's inequality we have that $\Pr(U<r^2)=1-\gamma$ implies that ${\mathrm{E}}U-(1-\gamma)^{-1/2}{\operatorname{sd}}U\le r^2\le{\mathrm{E}}U+\gamma^{-1/2} {\operatorname{sd}}U$. For ${\alpha}\le\Aupper$ the expectation ${\mathrm{E}}U_n({\alpha})$ is further bounded below by $C_{1,1} n^{-2{\alpha}/(1+2{\alpha}+2p)}$, for $C_{1,1}=C^{-4}(3^{1+2\Aupper+2p}+1)^{-1}$. Furthermore, ${\operatorname{sd}}U_n({\alpha})$ is bounded above by $C_{1,2,n}n^{-2{\alpha}/(1+2{\alpha}+2p)}$, for $C_{1,2,n}=\sqrt{10}C^4 n^{-(1/2)/(1+2\Aupper+2p)}\rightarrow0$. Hence, for large enough $n$ we have $C_{1,1}/2\geq(1-\gamma )^{-1/2}C_{1,2,n}$, whence (\[r\]) holds for $C_1^2= C_{1,1}/2$ and sufficiently large $n$. If $\Aupper$ depends on $n$, then so does $C_{1,1}=C_{1,1,n}$, but since $\Aupper\leq\sqrt{\log n}/4$ by assumption, we still have that $C_{1,2,n} \ll C_{1,1,n}$, and the preceding argument continues to apply. <span>Proof of (\[b\])</span> In view of the explicit expression for $\hat{\theta}_{\alpha}$ and (\[assumpinverse\]), $$\label{EqBiasSquare} \bigl{\Vert}B({\alpha};{\theta}_0)\bigr{\Vert}^2 =\sum_{i=1}^{\infty}\frac{i^{2+4{\alpha}}{\kappa}_i^{-4}{\theta}^2_{0,i}}{(i^{1+2{\alpha}}{\kappa}_i^{-2}+n)^2} \le C^8\sum_{i=1}^{\infty} \frac{i^{2+4{\alpha}+4p}{\theta}^2_{0,i}}{(i^{1+2{\alpha}+2p}+n)^2}.$$ The first term of the sum is zero following from our assumption $\theta _{0,1}=0$. Since the right-hand side of the preceding display is (termwise) increasing in ${\alpha}$, the supremum over ${\alpha}$ is taken at $\overline{\alpha}_n$. Because the map $x\mapsto x^{-1}\log x$ is decreasing for $x\ge e$ and takes equal values at $x=2$ and $x=4$, its minimum over $[2,N^{1+2{\alpha}+4p}]$ is taken at $N^{1+2{\alpha}+4p}$ if $N^{1+2{\alpha}+4p}\ge4$. Therefore, for $2\le i\le N$ we have that $i^{1+2{\alpha}+4p}\le N^{(1+2{\alpha}+4p)}\log i/\log N$ if $N\ge 4^{1/(1+2{\alpha}+4p)}$. Applied with $N_{\alpha}=n^{1/(1+2{\alpha}+2p)}$, this shows that, for $n\ge 4^{(1+2{\alpha}+2p)/(1+2{\alpha}+4p)}$, $$\sum_{2\le i \le N_{\alpha}}\frac{i^{2+4{\alpha}+4p}{\theta}^2_{0,i}}{(i^{1+2{\alpha}+2p}+n)^2} \le n^{-{2{\alpha}}/({1+2{\alpha}+ 2p})}h_n({\alpha};{\theta}_0).$$ This bounds the initial part of the series in the right-hand side of (\[EqBiasSquare\]). For ${\theta}_0\in\Theta_{pt}(L_0)$, the remaining part can be bounded above by $$\begin{aligned} &&\sum_{i\geq N_{\alpha}}\theta_{0,i}^2 \le L_{0}\sum_{i=N_{\alpha}}^{\rho N_{\alpha}} \theta_{0,i}^2 \leq L_{0}\bigl( \rho^{1+2{\alpha}+2p}+1\bigr)^2 h_n({\alpha}; {\theta}_0)N_{\alpha}^{-2{\alpha}},\end{aligned}$$ as is seen by lower bounding the series $h_n({\alpha};{\theta}_0)$ by the sum of its terms
{'source': 'arXiv_out/1310'}