On the validity of the formal Edgeworth expansion for
posterior densities

arXiv:1710.01871v1 [math.ST] 5 Oct 2017

John E. Kolassa
Department of Statistics and Biostatistics
Rutgers University
kolassa@stat.rutgers.edu
Todd A. Kuffner
Department of Mathematics
Washington University in St. Louis
kuffner@wustl.edu
October 6, 2017
Abstract
We consider a fundamental open problem in parametric Bayesian theory, namely the
validity of the formal Edgeworth expansion of the posterior density. While the study of
valid asymptotic expansions for posterior distributions constitutes a rich literature, the validity of the formal Edgeworth expansion has not been rigorously established. Several
authors have claimed connections of various posterior expansions with the classical Edgeworth expansion, or have simply assumed its validity. Our main result settles this open
problem. We also prove a lemma concerning the order of posterior cumulants which is of
independent interest in Bayesian parametric theory. The most relevant literature is synthesized and compared to the newly-derived Edgeworth expansions. Numerical investigations
illustrate that our expansion has the behavior expected of an Edgeworth expansion, and that
it has better performance than the other existing expansion which was previously claimed
to be of Edgeworth-type.
Keywords and phrases: Posterior; Edgeworth expansion; Higher-order asymptotics;
Cumulant expansion.

1

Introduction

The Edgeworth series expansion of a density function is a fundamental tool in classical asymptotic theory for parametric inference. Such expansions are natural refinements to first-order
asymptotic Gaussian approximations to large-sample distributions of suitably centered and
normalized functionals of sequences of random variables, X1 , . . . , Xn . Here, n is the available sample size, asymptotic means n → ∞, and first-order means that the approximation
1

using only the standard Gaussian distribution incurs an absolute approximation error of order
O(n−1/2 ). The term formal in conjunction with Edgeworth expansions means that derivation
of the expansion begins by expanding the log characteristic function, and then utilizes Fourier
inversion to obtain the corresponding density [Barndorff-Nielsen and Cox, 1979, p. 280]. The
coefficients of such expansions are expressed in terms of cumulants of the underlying density,
together with a set of orthogonal basis functions for a suitably general hypothesized function
space for the density being approximated. Asymptotic expansions are said to be valid if the
absolute approximation error incurred, as an order of magnitude in n, by truncating the series
expansion after a finite number of terms, is of the same asymptotic order as the first omitted term. The validity of the formal Edgeworth expansion is of foundational importance, in
the sense that this implies a certain degree of regularity of the statistical model, and the expansion itself offers deeper insights into the finite sample performance of many frequentist
inference procedures, such as those based on the likelihood. Applying standard arguments to
justify term-by-term integration of the truncated Edgeworth expansion for a density yields the
corresponding Edgeworth approximation for the cumulative distribution function. Such expansions are essential to studying the coverage accuracy of confidence sets, as well as establishing
higher-order relationships between different methods for constructing such approximate confidence sets. Our understanding of the bootstrap has also been greatly enhanced by studying
connections with Edgeworth expansions [Hall, 1992].
In contrast to central limit theorems in the frequentist context, the large-sample Gaussian
approximation of the posterior distribution of a suitably centered and normalized parameter is
typically justified using a Bernstein-von Mises theorem. Such theorems establish stochastic
convergence of the total variation distance between the sequence of posterior distributions and
an appropriate Gaussian distribution, where stochastic convergence is with respect to the true
distribution from which samples are independently drawn. The exact form depends on the
centering statistic and its corresponding variance estimate. For a lucid discussion, see van der
Vaart [1998, Ch. 10].
Somewhat surprisingly, the validity of formal Edgeworth expansion for the posterior density, arising from a Bayesian analysis, has not previously been established. This is less surprising when the challenging nature of this problem is understood. The term valid in the context
of posterior expansions means that, when approximating the posterior by truncating the series,
the absolute error is uniformly of the proper order on a set of parameter values whose posterior probability does not go to zero. While some authors have studied related expansions,
or made claims about the similarity of such expansions to classical Edgeworth expansions, to
our knowledge there is no existing proof of the validity of the formal Edgeworth series expansion for posterior distributions. Apart from formal Edgeworth expansion validity being of
foundational importance, we note that approximate posterior inference through higher-order
asymptotics remains of interest in parametric Bayesian theory; see, for example, Ruli et al.
[2014] or Kharroubi and Sweeting [2016]. Other approximate Bayesian inference procedures,
such as variational Bayes, or approximate Bayesian computation, have become popular due to
their ability to circumvent computationally expensive Markov chain Monte Carlo procedures.
Higher-order asymptotics offers another route to approximate Bayesian inference which, in
many settings of practical interest, can be extremely accurate and inexpensive to implement.
Within the existing literature on posterior expansions, there have been two dominant approaches to obtaining approximations of posterior quantities. The common starting point is
to express the posterior mean or density as a ratio of two integrals. Expansions for the nu2

merator and denominator separately, which may be truncated and integrated to yield integral
approximations, can yield valid expansions for the posterior quantity through formal division
of the numerator and denominator expansions. One approach which heavily emphasizes Taylor expansion is found in Johnson [1967, 1970] and Ghosh et al. [1982]. The most popular
approach, however, is to apply Laplace’s method to approximate the respective integrals, and
then use the ratio of these approximations; see Lindley [1961, 1980], Davison [1986], Tierney
and Kadane [1986]. Validity of Laplace expansions for posterior densities is considered in Kass
et al. [1990].
Another well-established method of posterior expansions utilizes Stein’s identity; see, for
instance, Woodroofe [1989, 1992], Weng [2003], Weng and Tsai [2008] and Weng [2010].
In this paradigm, Weng [2010] claimed to have established an Edgeworth expansion for the
posterior density. Compared to other expansions, Weng’s approach most closely resembles the
final form of an Edgeworth expansion in that it is expressed in terms of moments, but it is not
a formal Edgeworth expansion, and its structure is actually quite different from an Edgeworth
expansion, as we show below.
Bickel and Ghosh [1990], in a paper establishing Bartlett correctability of the posterior
distribution of the likelihood ratio statistic, implied that their regularity conditions would imply existence of Edgeworth expansions for the posterior of a particular functional, centered
and normalized by maximum likelihood quantities. However, they never actually claimed that
posterior Edgeworth expansions had been established as valid.
In fact, we will explain below that careful examination of their regularity conditions shows
that they simply assume the validity of a posterior Edgeworth expansion in order to establish
the validity of the posterior Bartlett correction. They do not actually prove that the Edgeworth
expansion is valid.
To establish validity of the formal Edgeworth expansion for a density, it is required to show
that the coefficients in the expansion, i.e. the cumulants of the statistical functional which is
being approximated, are of the proper asymptotic order to ensure that the terms of the expansion
have the claimed orders as powers of n−1/2 . This entails formally proving that power series
expansions for those cumulants are valid.
The existing results concerning validity of cumulant expansions are all within the sampling
distribution framework, and as such are not applicable to the Bayesian setting. Numerous
authors have studied expansions for posterior moments, but such results do not actually imply
that the corresponding cumulants are of the proper asymptotic order.
Edgeworth expansion relies on proper order for cumulants of the variable under investigation, after dividing by its standard deviation. These cumulants for the standardized variable
are known as invariant cumulants, and demonstrating their proper order is more delicate than
demonstrating the proper order for the underlying moments. As an example, consider the relationship between the posterior variance σ 2 and the fourth central moment µ04 . In order for the
formal Edgeworth expansion to have the proper asymptotic behavior, µ04 σ −4 −3 = O(1/n), and
so µ04 = σ 4 O(1/n). However, the converse – that µ04 = σ 4 O(1/n) implies µ04 σ −4 −3 = O(1/n)
– does not hold. Hence bounds on the moments (even for central moments) cannot guarantee
proper size of the invariant cumulants. Results exploring the parameter centered at something
only approximating the posterior mean (for example, the maximum likelihood estimator), and
standardized by something other than the exact posterior standard deviation (for example, an
approximation based on the Fisher information) will not ensure proper order for the invariant
cumulants.
3

In this paper, we make several novel contributions. First, we prove the validity of the
formal Edgeworth series expansion of the posterior density and distribution function. This
requires us to prove a lemma concerning the asymptotic order of posterior cumulants, which
is of independent interest, and appears to be the first rigorously established general result of
this type. We also synthesize the relevant literature on posterior expansions, giving rigorous
explanations of how existing Edgeworth-type expansions are not actually formal Edgeworth
expansions. Finally, we provide a numerical illustration of our results.

2
2.1

Background
Posterior Expansions

Before proceeding, we note that one could consider either analytic or stochastic expansions for
posterior densities. For an analytic expansion, the observed data sequence is viewed as a subsequence of a given, fixed infinite sequence of realizations. Deriving analytic expansions amounts
to showing that the posterior has certain asymptotic properties for a given, well-behaved infinite
sequence of observations. The stochastic expansion viewpoint asserts that such well-behaved
sequences occur with probability tending to one, with respect to the true data generating probability distribution. In this paper, we consider analytic approximations for a given well-behaved
infinite sequence of observations, though it would be possible to give analogous stochastic versions where O(·) terms are replaced by corresponding Op (·) terms; see Sweeting [1995] and
Kass et al. [1990].
From a formal Edgeworth series perspective, existing posterior expansions are centered at
the wrong place, typically either the maximum likelihood estimator or true parameter value,
instead of the posterior mean or something which is approximating the posterior mean. As
noted by DasGupta [2008, §20.8], the maximum likelihood estimator and posterior mean are
closely related. Suppose that one observes a sequence of observations X (n) = (X1 , . . . , Xn ),
each of which are identical copies of a random variable X whose distribution Pθ depends on
a scalar parameter θ. Write Pθ0 for the distribution corresponding to the true value θ0 under
which each component of X (n) is generated. Let E(θ|X (n) ) denote the posterior mean under
some prior density, and let θ̂n be the maximum likelihood estimator for θ based on X (n) . Under
standard regularity conditions, E(θ|X (n) ) − θ̂n , and also n1/2 (E(θ|X (n) ) − θ̂n ) converge in
Pθ0 -probability to zero. Therefore,
E(θ|X (n) ) − θ̂n = op (n−1/2 )

(1)

under Pθ0 . The effects of centering in the wrong place are examined in § 4. In particular, we
discuss the expansions given by Weng [2010] and Hartigan [1965]. These two expansions are
not Edgeworth series expansions, but for reasons explained in § 4, these can be considered to
have the closest relationship to our formal Edgeworth expansions.
It may appear strange to the reader that we are claiming the Edgeworth expansion for the
posterior has not been established as valid, even under regularity conditions common in the
literature. After all, there is the celebrated posterior Bartlett correction of Bickel and Ghosh
[1990], and the conventional derivation of the validity of the Bartlett correction requires a valid
Edgeworth expansion. Some authors, e.g. Chang and Mukerjee [2006], refer to the Bickel
and Gosh regularity conditions on the Bayesian model specification [Bickel and Ghosh, 1990,
4

p. 1078] as Edgeworth assumptions. Indeed, Bickel and Ghosh [1990] simply assume that an
Edgeworth expansion exists, but do not prove that it is valid. In particular, condition Bm (iv)
in Bickel and Ghosh [1990, p. 1079] is made to ensure that, when approximating the mean
of the loglikelihood statistic by the coefficient on the n−1 term in the expansion of that mean,
the remainder term in the approximation is of a small enough order. A similar assumption is
condition (i) of Bickel et al. [1985, p. 755]. Such assumptions are tantamount to assuming
that the Edgeworth expansion for the posterior is valid, which in turn implies that the cumulant
expansions are assumed to be valid. Therefore, the validity of the Edgeworth expansion is
assumed in these papers, but not formally proven. Moreover, the methods of derivation and
resulting terms of these expansions disqualify them from consideration as formal Edgeworth
expansions.
Related to the approach of Bickel and Ghosh [1990], some authors use an expansion for
the loglikelihood at the maximum likelihood estimate to obtain a posterior expansion which
has some structure resembling an Edgeworth expansion; see Datta and Mukerjee [2004, Eq.
2.2.19]. As with other expansions mentioned above, this is not an Edgeworth expansion for
several reasons. First, it is not derived by formal expansion of the posterior characteristic function. Second, the centering is at the maximum likelihood estimate, not the posterior mean.
Third, the first correction term is a linear one, which vanishes in an Edgeworth expansion.
Moreover, the coefficients are not cumulants of the posterior. We further note that in Bickel
and Ghosh [1990] and Datta and Mukerjee [2004, Lemma 4.2.1], an approximation is given
for the posterior characteristic function loglikelihood ratio statistic. DiCiccio and Stern [1993]
consider approximation of the posterior moment generating function of this statistic. All of
these expansions rely on regularity conditions which amount to assuming the validity of the
Edgeworth expansion, though none of these papers contain proofs, nor do they actually use formal Edgeworth expansions in their arguments. A lucid discussion of frequentist and Bayesian
Bartlett correction, and where the assumption of validity of Edgeworth expansions is essential,
is found in DiCiccio and Stern [1994].

2.2

Validity and Formal Edgeworth Expansions for Sampling Distributions

In the frequentist context, asymptotic expansions and their components are studied with respect
to the sampling distribution of the relevant statistical functional, under hypothetical repeated
sampling. Wallace [1958] provided the conventional notion of validity for an asymptotic expansion. Suppose
each function gn (y) in a sequence {gn }n≥1 is approximated by any partial
P∞
sum of a series j=0 n−j/2 Aj (y), where the Aj (·) do not depend on n. If for some constant
Cr (y), the absolute errors satisfy
r
X
gn (y) −
n−j/2 Aj (y) ≤ n−(r+1) Cr (y),
j=0

then the asymptotic expansion is said to be valid to r terms. If the constant Cr (y) does not
depend on y, then the asymptotic expansion is called uniformly valid in y. Hence, validity
requires that the absolute error in the approximation, using any partial sum, is of the same
order of magnitude as the first neglected term.
Consider a scalar random variable X with characteristic function γX (t) = E exp(itX), and
denote by X (n) = (X1 , . . . , Xn ) a sequence of independent and identically distributed copies
5

of X. Suppose that it is required to approximate the density gn (y) of Yn = n1/2 s(X1 , . . . , Xn )
for some scalar-valued function s(·), such that Yn is a centered and scaled statistic possessing
an asymptotically standard normal distribution to first order. The formal Edgeworth expansion
of the density gn (y) is derived according to the following steps; see, e.g. Jensen [1995, §1.5],
McCullagh [1987, Ch. 5], Hall [1992, Ch. 2], or Kolassa [2006, Ch. 3]. First, Taylor expand the
cumulant generating function of Yn , log γYn (t), in a neighborhood of zero, |t| < cn1/2 for some
c > 0. Next, expand the Fourier inversion integral over the region |t| < cn1/2 . Then, obtain a
bound on the inversion integral over the region |t| > cn1/2 . If Yn satisfies the assumptions of
the smooth function model [Hall, 1992, §2.4], then one can follow the program in the references
above to rigorously establish the validity of the Edgeworth expansion for gn (y). Other standard
references for Edgeworth series expansions include Feller [1971], Bhattacharya and Ghosh
[1978], Bhattacharya and Rao [2010] and Ghosh [1994].
To ensure that the formal Edgeworth series expansion for the density of Yn is valid in the
sense of Wallace [1958], it is required that the jth cumulant of Yn , denoted by κj,n , is of order
n−(j−2)/2 , and may expanded in a power series in n−1 :
κj,n = n−(j−2)/2 (cj,1 + n−1 cj,2 + n−2 cj,3 + · · · ),

j ≥ 1.

(2)

Since Yn is centered and scaled so that κ1,n = E(Yn ) → 0 and κ2,n = var(Yn ) → 1, then
c1,1 = 0 and c2,1 = 1. The origins of this result in the frequentist, repeated sampling setting
can be traced to the combinatorial arguments of James [1955, 1958], James and Mayne [1962]
and Leonov and Shiryaev [1959]. The interested reader is referred to Withers [1982, 1984],
McCullagh [1987, Chapter 2], Hall [1992, Chapter 2], Mykland [1999], Kolassa [2006], and
Stuart and Ord [1994, Chapters 12 and 13] for more details about cumulant expansions.
It is also of interest to integrate the Edgeworth series expansion of the density of Yn to
obtain an Edgeworth expansion for its corresponding distribution function.
Analogously to the density setting, this expansion is desired to be valid for fixed j as
n → ∞, and the remainder should be of the stated order uniformly in y. Sufficient regularity conditions [Hall, 1992, §2.2] for the validity of this expansion to order j are that
E(|X|j+2 ) < ∞ and
lim sup |γX (t)| < 1.
(3)
|t|→∞

The latter condition is known as Cramér’s condition.

2.3

Conditional Expansions and Posterior Expansions

In the frequentist sampling distribution framework concerning expansions for conditional densities, one might approach the problem by writing the conditional density as the ratio of a joint
density to a marginal density. After deriving Edgeworth expansions for the numerator and
denominator, formal division of these series expansions yields what is referred to as a directdirect Edgeworth expansion for the conditional density [Barndorff-Nielsen and Cox, 1989, Ch.
7]. Proving validity for these direct-direct expansions requires the analogous proofs of validity
for expansions of conditional cumulants, which are in general very difficult. Such direct-direct
expansions are not the same as a direct expansion of a conditional density, but the bigger obstacle to their utility is that they are non-Bayesian in nature. Standard sampling distribution
arguments do not apply when deriving an Edgeworth expansion for the posterior density of
ϑn = n1/2 (θ − θ0 )/σ, where θ0 and σ are the posterior expectation and standard deviation. In
6

particular, in this posterior setting, one does not have independent and identically distributed
θs, but rather a single θ. Furthermore, posterior inference is conditional on a single data set,
without appealing to repeated sampling arguments. An obvious point worth emphasizing is that
the consideration of increasingly larger sample sizes is not the same as considering hypothetical
repeated sampling. As discussed above, we are assuming that the data represent a subsequence
from a fixed infinite sequence, rather than repeated random samples from a probability distribution.
A major obstacle to proving validity of posterior Edgeworth expansions is the issue of the
cumulant orders. In the sampling distribution framework, there are well-known results concerning the relationship between conditional and unconditional cumulants, but these results are
unfortunately of no use in the posterior framework. In particular, Brillinger [1969] established
a theorem which permits computation of unconditional cumulants from conditional cumulants;
see also Speed [1983] and McCullagh [1987, §2.9 and §5.6]. However, there is no converse to
Brillinger’s theorem, and even if there were, one must still overcome the issue that θ is a single random variable, rather than a sequence of independent and identically distributed random
variables.
Due to the challenges just mentioned, there are no general results about posterior cumulants
available in the literature. Pericchi et al. [1993] give some specific results regarding the form
of the cumulant generating function only relevant to exponential families. Hartigan [1965, p.
1145] alluded to the order of posterior cumulants, but was not precise about how such orders
could be shown.

3

Main results

We consider expansions for the posterior distribution of the scalar-valued quantity ϑn = n1/2 (θ−
θ0 )/σ, for θ0 and σ the posterior expectation and standard deviation. It is assumed throughout
that an appropriate Bernstein-von Mises theorem holds for the sequence of posterior distributions of ϑn . Due to the large variety of asymptotic normality results in the literature, and since
our goal is to prove validity of the Edgeworth expansion in some generality, we do not discuss
all of the conditions needed for the various specific Bernstein-von Mises theorems to hold. Our
treatment of regularity conditions focuses on those conditions of particular relevance to the
validity of the formal Edgeworth expansion.
The first step in our analysis is to prove that the invariant posterior cumulants admit a valid
power series expansion, establishing that the coefficients in the Edgeworth series expansion
will have the correct asymptotic order. We then prove validity of the expansion for the posterior
density and distribution function, respectively.

3.1

The Order of Posterior Cumulants

Given a sequence X (n) = (X1 , . . . , Xn ) of n random variables which, conditional on the value
of a scalar random parameter θ taking values in a set Θ ⊂ <,
independent and identically
Qare
n
distributed according to density f (x|θ), define Ln (θ; x) = i=1 f (xi |θ) to be the likelihood
function. Here, x represents the observations of the sequence X (n) . Denote the loglikelihood
function by `n (θ) ≡ `n (θ; x). Assume that, prior to observing the data, the uncertainty about θ

7

is described by a prior density function π(θ). The posterior density of θ is defined as
Q
π(θ) ni=1 f (xi |θ)
Ln (θ; x)π(θ)
Qn
=R
.
f (θ|x) = R
π(θ) i=1 f (xi |θ)dθ
L (θ; x)π(θ)dθ
Θ
Θ n
Throughout, we suppress the dependence on x when there is no chance of confusion.
Lemma 1. Assume that the likelihood function has a single global maximizer θ̂n . Define the
average loglikelihood `¯n (θ) = `(θ)/n, and assume that `¯n and the log prior density have six
continuous derivatives in a neighborhood of the form θ̂n ±  for  independent of n, and such
that `¯n (θ) < `¯n (θ̂n ) − δ for θ ∈
/ (θ̂n − , θ̂n + ), and assume that the second derivative of
the average loglikelihood is bounded away from zero on this neighborhood. Then the invariant
cumulant of θ of order j, defined to be the cumulant of order j of (θ − θ0 )/σ, and denoted by
κj , is O(n(2−j)/2 ) for j ∈ {3, 4, 5}. Here θ0 and σ are the expectation and standard deviation
of the posterior distribution, respectively.
Proof. Arguments in the first half of this proof hold for loglikelihood functions and log prior
densities with varying numbers of derivatives; denote this number by k, and until specified
otherwise, it might, but need not be, 5. By (1), there exists N (potentially dependent on the
sample) so that for n ≥ N , |θ0 − θ̂n | < /2, and so continuous derivatives to order k exist for
`¯n (θ) and the log prior density at θ0 . Hence the log prior has an expansion
−

k
X

hj (θ − θ0 )j /j! + Q(θ)(θ − θ0 )k+1 /(k + 1)!,

j=0

and the loglikelihood has an expansion
−n[

k
X

gj (θ − θ0 )j /j! + Q∗ (θ)(θ − θ0 )k+1 /(k + 1)!],

j=0

where the coefficients gj and hj may be calculated from derivatives of the loglikelihood and
log prior, respectively. Here Q(θ) and Q∗ (θ) are the standard Taylor series remainder terms,
calculated from the derivatives of order k +1 of the log prior density and average loglikelihood,
respectively, evaluated at a parameter value intermediate between θ0 and θ.
The log posterior can be expressed as
k
X

pj (θ − θ0 )j /j! + [nQ∗ (θ) + Q(θ)],

j=0

where pj = −hj − ngj . The first term p0 may be chosen to make the posterior integrate to 1.
The error terms Q(θ) and Q∗ (θ) may be taken as bounded for θ ∈ (θ̂n − /2, θ̂n + /2). The
choice of θ0 ensures that p1 = O(n1/2 ).
Let ω(θ) be the polynomial
resulting from retaining only terms of order k and smaller in the
Pk
power series for exp( j=3 pj (θ − θ0 )j ). Let µ∗ represent the extended Laplace approximation
to the posterior moments; that is,
Z ∞
1
∗
µj =
exp(−p0 ) exp(− (θ − θ0 )2 p2 )θj ω(θ) dθ.
2
−∞
8

Let µj denote the true values of these moments. Choose  > 0 to satisfy the conditions of
the lemma such that both
| log(π(θ0 )) + n`¯n (θ0 ) − log(π(θ)) − n`¯n (θ)| ≤ p2 (θ − θ0 )2 /4,
and
|

k
X

pj (θ − θ0 )j | ≤ p2 (θ − θ0 )2 /2,

j=3

for |θ − θ0 | < . Since the difference between the maximum likelihood estimator and the
posterior expectation is Op (1/n1/2 ), then there exists δ > 0 such that `¯n (θ) < `¯n (θ0 )−δ for θ ∈
(−/2, /2)c , and the contribution from outside of the interval to the absolute approximation
error |µ∗j − µj | is bounded by exp(−nδ). Inside (−/2, /2), the contribution to |µj − µ∗j | is
bounded by


!k+1
k
X
1
/(k + 1)! ,
exp(−p0 ) exp(−n (θ − θ0 )2 p2 ) |nQ∗ (θ) + Q(θ)| +
pj (θ − θ0 )j
4
j=3
by Kolassa [2006, Theorem 2.5.3], which is O(n−(k+1)/2 ).
Take k = 5. In this case,
ω(θ) = 1 − p1 (θ − θ0 ) + p22 (θ − θ0 )2 /2 − (p31 + p3 )(θ − θ0 )3 /6
+(p41 + 4p3 p1 − p4 )(θ − θ0 )4 /24
−(θ − θ0 )5 (p51 + 10p3 p21 − 5p4 p1 + p5 )/120.
Moments approximated in this way are accurate to O(n−7/2 ), and cumulants approximated
using standard formulas for producing cumulants from moments [Kolassa, 2006, p. 10] are
accurate to the same order. Denote the cumulant of θ of order j by βj . The first cumulant is
β1 = g1 g2−1 + (g1 h2 g2−2 − h1 g2−1 )n−1 + O(1/n2 ). Recall that the gj terms are the coefficients
in the expansion of the average loglikelihood about the posterior mean. They are all O(1)
except g1 . Since the posterior mean is within O(n−1/2 ) of the maximum likelihood estimate,
then the choice of θ0 as the posterior mean forces g1 = O(n−1/2 ). The second cumulant is
β2 = 21 g2−1 n−1 + O(n−2 ), the third cumulant is β3 = − 16 g3 g2−3 n−2 + O(n−3 ), and the fourth
1
cumulant is β4 = − 24
(2g32 g2−5 −g4 g2−4 )n−3 +O(n−7/2 ). The most delicate part of the argument
is the calculation of these cumulants, and ensuring that larger terms cancel to leave a remainder
of the proper order. The proof is then completed by dividing the cumulants by the proper
power of the second cumulant, β2 , which is O(n−1 ), to see that the quotient is of the proper
j/2
order. Specifically, the invariant cumulant of order j, j ≥ 3, is κj = βj /β2 . Since g2 , g3 , g4
are bounded away from zero, the invariant cumulants of order 3 and 4 are O(n−1/2 ) and O(n−1 )
respectively. Similar calculations show that the invariant cumulant of order 5 is O(n−3/2 ).
Remark 1. The argument above holds to provide bounds on moments of all orders. Orders of
cumulants are more delicate, since cumulants are expressed in terms of differences of products
of moments, and proper order for cumulants requires that leading terms of the representation
properly cancel. At present we know of no way to do this except on a case-by-case basis. This
difficulty extends to bounds on invariant cumulants.
9

Remark 2. In § 1 we argued that one cannot bound the cumulants based on the order of the
moments. We are instead bounding cumulants by getting a Laplace expansion for the moments,
and observing that enough leading terms cancel to show that the cumulants are of the proper
order.
Remark 3. As one would expect, the hj terms, corresponding to coefficients in the expansion
of the log prior about the posterior mean, appear only in terms of order n−1 and smaller.

3.2

Validity of Edgeworth expansion for the posterior density

Theorem 1. Let A be a subset of the sample space. Suppose that for any x ∈ A, the following
assumptions hold.
1. The prior is absolutely continuous with respect to Lebesgue measure.
2. The likelihood function Ln (θ; x) is a measurable function of θ.
3. The posterior is proper, and, for sufficiently large n, has a bounded density.
4. The likelihood function has a unique global maximizer θ̂n (x),
5. The loglikelihood `n (θ) is k-times differentiable in a neighborhood of θ̂n (x), with average
second derivative `00n (θ(x), x)/n bounded away from zero, and average jth derivative
(j)
`n (θ(x), x)/n bounded, for 2 ≤ j ≤ k.
Define ϑn = n1/2 (θ − θ0 )/σ, with θ0 the posterior mean and σ the standard deviation of the
posterior distribution for θ. Let
ek,n (ϑ) = φ(ϑ){1 + h3 (ϑ)κ3 /6 + κ4 h4 (ϑ)/24 − κ23 h6 (ϑ)/72 + · · · },

(4)

where κj are cumulants of ϑn , and hence the invariant
Pej k,n is truncated to
Qj cumulants of θ, and
contain only terms with products of κj of the form m=1 κrm , such that m=1 (rm − 2) < k.
Heuristically, ek,n contains terms of size larger than O(n−k/2 ), where κj satisfies
κj = O(n−(j−2)/2 ),

j ≥ 3.

(5)

Then the error in the use of ek,n (ϑ) to approximate the posterior density is of order O(n−k/2 ),
uniformly in ϑ and uniformly in x in a compact subset of the sample space, but not relatively.
R∞
Proof. Let λ(τ ) = −∞ Ln (θ)π(θ) exp(iθτ ) dθ. The characteristic function for θ is then
λ(τ )/λ(0), and the characteristic function for ϑn = n1/2 (θ − θ0 )/σ is
ϕ(τ ) = λ(n1/2 τ ) exp(−n1/2 θ0 τ i)/λ(0).

(6)

Here σ is the standard deviation for the posterior distribution for θ.
The Riemann–Lebesgue
theorem, using Assumption 3 above, indicates that |λ(τ )| ≤ C/|τ |
R∞
for C = 2 −∞ Ln (θ)π(θ) dθ; see Billingsley [1995, Theorem 26.1]. Furthermore, by Assumption 3, there exist m and C1 and C2 such that
Z ∞
|λ(τ )|m dτ ≤ C1
and hence
|ϕ(τ )| ≤ C2 /(n1/2 |τ |).
(7)
−∞

10

Then, in parallel with the development of Feller [1971, §XV.3], the Fourier inversion of ϕ to
obtain ek,n (ϑ) as the posterior density of ϑn is performed by first expanding φ in θ near θ̂. The
Fourier inversion integral resultsR approximately in ek,n .
∞
More formally, let γn (ζ) = −∞ exp(iζϑ)ek,n (ϑ) dζ, for ek,n (ϑ) defined in (4). In parallel
with the development of Feller [1971, §XVI.2], the posterior density for ϑn is given by
Z ∞
1
exp(−iζϑ)ϕ(ζ) dζ,
(8)
2π −∞
and the difference between the true posterior density and the Edgeworth series approximation
of (4) is bounded by
Z ∞
1
exp(−iζϑ)|ϕ(ζ) − γn (ζ)| dζ
2π −∞
Z
1
=
exp(−iζϑ)|ϕ(ζ) − γn (ζ)| dζ
2π (−δn1/2 ,δn1/2 )
Z
1
exp(−iζϑ)|ϕ(ζ) − γn (ζ)| dζ. (9)
+
2π (−δn1/2 ,δn1/2 )c
As discussed by Kolassa [2006, §3.7], the first of these integrals is bounded by
Z ∞
1
exp(−iζϑ)p(ζ, n)/nk/2 dζ,
2π −∞
for p(ζ, n) a polynomial in ζ and 1/n1/2 . This polynomial has coefficients that depend on
derivatives of the loglikelihood, and so the error is uniformly of the proper order. Note that
result (7) applies to γn (ζ) as well as to ϕ(ζ); choose the resulting constants m∗ ≥ m, C1∗ ≥ C1 ,
and C2∗ ≥ C2 . These together show that the second integral in (7) is bounded by
∗

(C2∗ /(δn1/2 ))n−m C1∗ n/δ,
which is geometrically small.

3.3

Validity of Edgeworth expansion for the posterior distribution function

By assuming that the prior is a density, and that the likelihood is continuously differentiable, we
actually have more smoothness than is required for Cramér’s condition (3) to hold. However,
as noted above, the extra smoothness implied by these assumptions is necessary to prove the
validity of the cumulant expansions in Lemma 1.
Theorem 2. Under the assumptions of Theorem 1, define the Edgeworth approximation to the
posterior cumulative distribution function of ϑ as
Ek,n (ϑ) = Φ(ϑ) − φ(ϑ){h2 (ϑ)κ3 /6 + κ4 h3 (ϑ)/24 − κ23 h5 (ϑ)/72 + · · · }.

(10)

The absolute error incurred in using (10) to approximate the distribution function of ϑn is
uniformly of order O(n−3/2 )
Proof. As noted by Kolassa [2006, Equation (48)], the error in applying (10) to approximate
the posterior distribution function is given by the left side of (9), modified by dividing the integrand by |ζ|, in this case with a density. By the previous application of the Riemann-Lebesgue
theorem, the modified integral representing error converges absolutely, and is multiplied by the
proper power of the sample size.
11

4

Relationship to Existing Edgeworth-type Expansions

Weng [2010] provides an asymptotic expansion for the posterior distribution of a statistical
model for data consisting of n independent and identically distributed observations, satisfying
certain regularity conditions, by centering the distribution at the maximum likelihood estimate,
and scaling the difference between a potential parameter value and the estimate by the second derivative of the loglikelihood evaluated at the maximum likelihood. This produces an
asymptotic expansion valid to O(n−(s+1)/2 ), and uses 3s − 1 terms. For example, when approximating the posterior CDF, the approximation with error O(n−3/2 ) uses s = 2, and hence
uses five terms, including the leading term represented by the normal cumulative distribution
function. This expansion includes Hermite polynomials to order 5, as is found in the standard
Edgeworth expansion presented by, for example, McCullagh [1987]. However, since the maximum likelihood estimate is not the same as the posterior expectation, the leading term in the
Weng approximation does not match the target distribution as well as one centered at the true
posterior expectation. This lack of match leads to a more complicated expansion. Furthermore,
the example Weng presents provides finite sample performance that is inferior to that generally
expected from an approximation with asymptotic error O(n−3/2 ), as we illustrate in § 5.
We now demonstrate that Weng’s approximation has an error that is equivalent to that of our
approximation; these calculations also demonstrate the differences between the two approximatons. Weng’s approximation for the posterior distribution function of ϑ̃n = (θ − θ̂n )/σ̂, where
θ̂n is the maximum likelihood estimate and σ̂ is the square root of the observed information
evaluated at θ̂n , is of the form
P [(θ − θ̂)/σ̂ < ϑ̃n |x] = Φ(ϑ̃n ) −

3s
X

qi−1 (ϑ̃n )φ(ϑ̃n )ci ,

i=1

for qi Hermite polynomials, and ci constants given by Stein’s lemma. Weng [2010] shows that
this approximation holds uniformly for ϑ̃ ∈ <.
Hartigan [1965] also provides an approximation to the posterior, in this case to the density,
and obtains an approximation of a similar form. This approximation is also about a center
other than the posterior expectation; in this case, the expansion is in the neighborhood of a
true parameter value. The notation here is similar to that of Hartigan [1965]. Suppose data
X1 , . . . , Xn is observed, with observations independent and identically distributed, conditional
on a scalar
common log density g(xi |θ) and log prior density h(θ). Let
R j parameter θ,Pwith
n
ωj = Θ θ exp(h(θ) + i=1 g(xi |θ) dθ; these quantities depend on the data. The posterior
expectation is then ω1 /ω0 . Take s = 2; then Weng’s approximation to the posterior distribution
function is
Φ(ϑ̃n ) + φ(ϑ̃n ){c1 + ϑ̃n c2 + (ϑ̃2n − 1)c3 + (ϑ̃3n − ϑ̃n )c4 + (ϑ̃5n − 10ϑ̃3n + 15ϑ̃n )c6 },
and to the density is
w(ϑ̃n ) = φ(ϑ̃n ) − φ(ϑ̃n ){ϑ̃n c1 + (ϑ̃2n − 1)c2 + (ϑ̃3n − ϑ̃n )c3 + (ϑ̃4n − 6ϑ̃2n + 3)c4
+(ϑ̃6n − 15ϑ̃4n + 45ϑ̃2n − 15)c6 },
uniformly (in θ) to O(n−3/2 ). Then the expectation associated with this density approximation
is −c1 , the variance associated with this approximation is 1 − c21 − 2c2 , and the approximation
12

to the density d(ρ) of ρ = (θ − θ̂n )/σ̂ − c1 satisfies
d(ρ) = w(ρ(1 − c21 − 2c2 )1/2 − c1 )(1 − c21 − 2c2 )1/2 + O(n−3/2 ).

(11)

Weng [2010] notes that
c1 , c3 = O(n−1/2 ), c2 , c4 , c6 = O(n−1 ).

(12)

Expanding w(ρ(1 − c21 − 2c2 )1/2 − c1 )(1 − c21 − 2c2 )1/2 in terms of powers of n1/2 , and bounding
errors using, for example, Kolassa [2006, Theorem 2.5.3], one can exhibit w(ρ(1−c21 −2c2 )1/2 −
c1 )(1 − c21 − 2c2 )1/2 of the form
φ(ρ) − φ(ρ){ρc∗1 + (ρ2 − 1)c∗2 + (ρ3 − ρ)c∗3 + (ρ4 − 6ρ2 + 3)c∗4 +
(ρ6 − 15ρ4 + 45ρ2 − 15)c∗6 }
for constants c∗j satisfying (12), and furthermore, c∗1 = c∗2 = 0. Hence (11) is an Edgeworth
expansion to O(n−3/2 ).
Remark 4. We have given two proofs of the validity of Edgeworth expansion of the posterior.
The first is a direct proof for the formal Edgeworth expansion (Theorem 1), while the second
is not a formal Edgeworth expansion, but rather shows how to correct Weng’s expansion due
to using the wrong center. As the above arguments demonstrate, one can obtain an Edgeworthtype expansion by centering at the maximum likelihood estimate θ̂n and correcting. Such an
expansion could have the same form as an Edgeworth series, but would not be a formal Edgeworth expansion, and would require additional work to compute the correction factors.
Remark 5. We have used the Laplace approximation of the cumulants only to show that they
are of the correct asymptotic order. It is not necessary to use the Laplace approximation for
implementation of the expansion. In practice, any sufficiently accurate estimator of the posterior moments could be used to implement the Edgeworth expansion. One approach is given by
Hartigan [1965]. Another is to use the constants given by Weng and adjust them accordingly.
Remark 6. Weng’s expansion is actually more similar to a Gram-Charlier expansion of the
posterior, not an Edgeworth expansion. Weng ensures that the pseudo-moments are of the
correct order, but not the cumulants. For example, in her displayed equation (45), the set J2
(corresponding to the n−1 term in the expansion) includes the sixth Hermite polynomial and its
multiplier. For an Edgeworth expansion, this term is discarded.

5

Example

Consider a random variable having a binomial distribution, X ∼ Bin(θ, n) with a beta prior,
θ ∼ Beta(a, b). Suppose that a = 0.5, b = 4.0, n = 5 and x = 2. This example was previously
considered by Weng [2010], who, using Stein’s identity, derived an asymptotic expansion using up to 40 moments. This is an ideal example for illustrating the performance of posterior
expansions, because the sample size is small, and the normal approximation is inaccurate, due
to the skewness in the posterior.
Weng’s expansion, for the density of θ, is given in Figure 1. This figure should be compared
with Figure 2, showing our posterior Edgeworth approximation for the posterior density of θ.
Note that the standard Edgeworth approximation, with four moments, behaves better than the
13

Figure 1: Density approximation of Weng (2010).

3.0

Beta binomial model

2.0
1.5
1.0
0.5
0.0

Density Approximations

2.5

True
Weng 6 moments
Normal
Weng 40 moments

0.0

0.2

0.4

0.6

0.8

Parameter
Prior beta parameters (0.50,4.00); Successes 2 Number of Trials 5

14

1.0

Figure 2: Edgeworth approximation of the posterior density.

3.0

Beta binomial model

2.0
1.5
1.0
0.5
0.0

Density Approximations

2.5

True
Edgeworth 3 moments
Edgeworth 4 moments
Normal

0.0

0.2

0.4

0.6

0.8

Parameter
Prior beta parameters (0.50,4.00); Successes 2 Number of Trials 5

15

1.0

Stein’s identity approach using 40 moments. To understand this phenomenon, consider the
formal Edgeworth derivation due to Davis [1976] and presented by McCullagh [1987]. When
constructing an approximation of a density f around a baseline density g, obtain the formal
series
∞
X
f (x) = g(x)
hj (x)µ∗j /j!,
j=0

where the functions hj are ratios of derivatives of g to g itself, and the coefficients µ∗j represent
the results of calculating differences in cumulants between f and g, and applying the standard relationship giving moments from cumulants to these cumulant differences to get pseudocumulants. Standard Edgeworth approximation techniques and the method of Weng [2010]
use as g a normal density; standard Edgeworth approximations use g with mean and variance
matching f . When applying Edgeworth techniques to a posterior, then, the standard approach
is to match the mean and variance. Weng [2010] uses instead the maximum likelihood estimate
and its usual standard error, and so convergence is slower.
Figure 3 displays the absolute error the normal approximation, and the Edgeworth approximation of the posterior density.
Figure 3: Absolute error of density approximations.
Beta binomial model

0.0
−0.2
−0.4
−0.6

Error in Density Approximations

0.2

0.4

Edgeworth 3 moments
Edgeworth 4 moments
Normal

0.0

0.2

0.4

0.6

0.8

Parameter
Prior beta parameters (0.50,4.00); Successes 2 Number of Trials 5

16

1.0

Acknowledgement
Both authors were partially supported by the National Science Foundation. The second author is grateful to J. K. Ghosh, Jens Jensen, Trevor Sweeting and Alastair Young for helpful
correspondence and discussion related to this topic during 2010-2011.

References
O. Barndorff-Nielsen and D. R. Cox. Edgeworth and saddle-point approximations with statistical applications. Journal of the Royal Statistical Society. Series B, 41(3):279–312, 1979.
O. E. Barndorff-Nielsen and D. R. Cox. Asymptotic Techniques for Use in Statistics. Chapman
& Hall, 1989.
R. N. Bhattacharya and J. K. Ghosh.
On the validity of the formal Edgeworth expansion.
Ann. Statist., 6(2):434–451, 1978.
ISSN 0090-5364.
URL
http://links.jstor.org/sici?sici=0090-5364(197803)6:2<434:
OTVOTF>2.0.CO;2-M&origin=MSN.
Rabi N. Bhattacharya and R. Ranga Rao. Normal Approximation and Asymptotic Expansions. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, updated and correction version of 1976 original. edition, 2010. ISBN 978-0-898718-973. doi: 10.1137/1.9780898719895.ch1. URL http://dx.doi.org/10.1137/1.
9780898719895.ch1.
Peter J. Bickel and J. K. Ghosh. A decomposition for the likelihood ratio statistic and the
Bartlett correction—a Bayesian argument. Ann. Statist., 18(3):1070–1090, 1990. ISSN
0090-5364. doi: 10.1214/aos/1176347740. URL http://dx.doi.org/10.1214/
aos/1176347740.
Peter J. Bickel, F. Götze, and W. R. van Zwet. A simple analysis of third-order efficiency of
estimates. In L.M. Le Cam and R.A. Olshen, editors, Proceedings of the Berkeley conference
in honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983), pages 749–768.
Wadsworth, Belmont, CA, 1985.
Patrick Billingsley. Probability and Measure. John Wiley & Sons, Inc., New York, 3rd edition,
1995. ISBN 0-471-00710-2. A Wiley-Interscience Publication.
D. R. Brillinger. The calculation of cumulants via conditioning. Annals of the Institute of
Statistical Mathematics, 21:215–218, 1969.
In Hong Chang and Rahul Mukerjee. Probability matching property of adjusted likelihoods.
Statist. Probab. Lett., 76(8):838–842, 2006. ISSN 0167-7152. doi: 10.1016/j.spl.2005.10.
015. URL http://dx.doi.org/10.1016/j.spl.2005.10.015.
Anirban DasGupta. Asymptotic Theory of Statistics and Probability. Springer, 2008.
Gauri Sankar Datta and Rahul Mukerjee. Probability Matching Priors: Higher Order Asymptotics. Springer-Verlag, 2004. ISBN 0-387-20329-X. doi: 10.1007/978-1-4612-2036-7.
URL http://dx.doi.org/10.1007/978-1-4612-2036-7.
17

A. W. Davis. Statistical distributions in univariate and multivariate Edgeworth populations.
Biometrika, 63(3):661–670, 1976. ISSN 0006-3444. doi: 10.1093/biomet/63.3.661. URL
http://dx.doi.org/10.1093/biomet/63.3.661.
A. C. Davison. Approximate predictive likelihood. Biometrika, 73(2):323–332, 1986.
T. J. DiCiccio and S. E. Stern. On Bartlett adjustments for approximate Bayesian inference.
Biometrika, 80(4):731–740, 1993.
T. J. DiCiccio and S. E. Stern. Frequentist and Bayesian Bartlett correction of test statistics
based on adjusted profile likelihoods. Journal of the Royal Statistical Society. Series B, 56
(2):397–408, 1994.
William Feller. An Introduction to Probability Theory and its Applications. Vol. II. 2nd. John
Wiley & Sons, Inc., New York-London-Sydney, 1971.
J. K. Ghosh, B. K. Sinha, and S. N. Joshi. Expansions for posterior probability and integrated
Bayes risk. In S. Gupta and J. Berger, editors, Statistical Decision Theory and Related Topics,
III, Vol. 1 (West Lafayette, Ind., 1981), pages 403–456. Academic Press, New York-London,
1982.
J.K. Ghosh. Higher-Order Asymptotics. Hayward: Institute of Mathematical Statistics, 1994.
Peter Hall. The Bootstrap and Edgeworth Expansion. Springer Series in Statistics. SpringerVerlag, New York, 1992. ISBN 0-387-97720-1. doi: 10.1007/978-1-4612-4384-7. URL
http://dx.doi.org/10.1007/978-1-4612-4384-7.
J. A. Hartigan. The asymptotically unbiased prior distribution. Ann. Math. Statist., 36:1137–
1152, 1965. ISSN 0003-4851. doi: 10.1214/aoms/1177699988. URL http://dx.doi.
org/10.1214/aoms/1177699988.
G. S. James. Cumulants of a transformed variate. Biometrika, 42:529–531, 1955. ISSN
0006-3444. doi: 10.1093/biomet/42.3-4.529. URL http://dx.doi.org/10.1093/
biomet/42.3-4.529.
G. S. James. On moments and cumulants of systems of statistics. Sankhyā, 20:1–30, 1958.
ISSN 0972-7671.
G. S. James and Alan J. Mayne. Cumulants of functions of random variables. Sankhyā Ser. A,
24:47–54, 1962. ISSN 0581-572X.
Jens Ledet Jensen. Saddlepoint Approximations. Oxford University Press, 1995. ISBN 0-19852295-9. Oxford Science Publications.
R. A. Johnson. An asymptotic expansion for posterior distributions. Ann. Math. Statist., 38:
1899–1906, 1967. ISSN 0003-4851. doi: 10.1214/aoms/1177698624. URL http://dx.
doi.org/10.1214/aoms/1177698624.
Richard A. Johnson. Asymptotic expansions associated with posterior distributions. Ann.
Math. Statist., 41:851–864, 1970. ISSN 0003-4851. doi: 10.1214/aoms/1177696963. URL
http://dx.doi.org/10.1214/aoms/1177696963.
18

Robert E. Kass, Luke Tierney, and Joseph P. Kadane. The validity of posterior expansions
based on Laplace’s method. In S. Geisser, J. S. Hodges, S. J. Press, and A. Zellner, editors,
Bayesian and Likelihood Methods in Statistics and Econometrics, pages 473–488. Elsevier
Science Publishers B.V., 1990.
S. A. Kharroubi and T. J. Sweeting. Exponential tilting in Bayesian asymptotics. Biometrika,
103(2):337–349, 2016. ISSN 0006-3444. doi: 10.1093/biomet/asw018. URL http://
dx.doi.org/10.1093/biomet/asw018.
John E. Kolassa. Series Approximation Methods in Statistics. Springer-Verlag, 3rd edition,
2006. ISBN 0-387-94277-7. doi: 10.1007/978-1-4757-4275-6. URL http://dx.doi.
org/10.1007/978-1-4757-4275-6.
V. P. Leonov and A. N. Shiryaev. On a method of semi-invariants. Theor. Probability Appl., 4:
319–329, 1959.
D. V. Lindley. The use of prior probability distributions in statistical inference and decisions. In
Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I, pages 453–468. Univ. California
Press, Berkeley, Calif., 1961.
Dennis V Lindley. Approximate Bayesian methods. Trabajos de Estadı́stica y de Investigación
Operativa, 31(1):223–245, 1980.
Peter McCullagh. Tensor Methods in Statistics. Chapman & Hall, London, 1987. ISBN 0-41227480-9.
Per Aslak Mykland. Bartlett identities and large deviations in likelihood theory. Ann. Statist.,
27(3):1105–1117, 1999. ISSN 0090-5364. doi: 10.1214/aos/1018031270. URL http:
//dx.doi.org/10.1214/aos/1018031270.
L. R. Pericchi, B. Sansó, and A. F. M. Smith. Posterior cumulant relationships in Bayesian
inference involving the exponential family. J. Amer. Statist. Assoc., 88(424):1419–
1426, 1993. ISSN 0162-1459. URL http://links.jstor.org/sici?sici=
0162-1459(199312)88:424<1419:PCRIBI>2.0.CO;2-0&origin=MSN.
Erlis Ruli, Nicola Sartori, and Laura Ventura. Marginal posterior simulation via higher-order
tail area approximations. Bayesian Anal., 9(1):129–145, 2014. ISSN 1936-0975. doi: 10.
1214/13-BA851. URL http://dx.doi.org/10.1214/13-BA851.
T. P. Speed. Cumulants and partition lattices. Australian Journal of Statistics, 25:378–388,
1983.
Alan Stuart and J. Keith Ord. Kendall’s Advanced Theory of Statistics Vol. 1. John Wiley &
Sons, 6th edition, 1994. ISBN 0-340-61430-7.
Trevor J. Sweeting. A framework for Bayesian and likelihood approximations in statistics.
Biometrika, 82(1):1–23, 1995. ISSN 0006-3444. doi: 10.1093/biomet/82.1.1. URL http:
//dx.doi.org/10.1093/biomet/82.1.1.
Luke Tierney and Joseph B Kadane. Accurate approximations for posterior moments and
marginal densities. Journal of the American Statistical Association, 81(393):82–86, 1986.
19

A. W. van der Vaart. Asymptotic Statistics. Cambridge University Press, 1998.
David L. Wallace. Asymptotic approximations to distributions. Ann. Math. Statist., 29:635–
654, 1958. ISSN 0003-4851. doi: 10.1214/aoms/1177706528. URL http://dx.doi.
org/10.1214/aoms/1177706528.
Ruby C. Weng. On Stein’s identity for posterior normality. Statist. Sinica, 13(2):495–506,
2003. ISSN 1017-0405.
Ruby C. Weng. A Bayesian Edgeworth expansion by Stein’s identity. Bayesian Anal., 5(4):
741–763, 2010. ISSN 1936-0975. doi: 10.1214/10-BA526. URL http://dx.doi.org/
10.1214/10-BA526.
Ruby C. Weng and Wen-Chi Tsai. Asymptotic posterior normality for multiparameter problems. J. Statist. Plann. Inference, 138(12):4068–4080, 2008. ISSN 0378-3758. doi: 10.1016/
j.jspi.2008.03.034. URL http://dx.doi.org/10.1016/j.jspi.2008.03.034.
C. S. Withers. Second order inference for asymptotically normal random variables. Sankhyā
Series B, 44(1):19–27, 1982.
C. S. Withers. Asymptotic expansions for distributions and quantiles with power series cumulants. Journal of the Royal Statistical Society. Series B, 46(3):389–396, 1984.
Michael Woodroofe. Very weak expansions for sequentially designed experiments: linear models. Ann. Statist., 17(3):1087–1102, 1989. ISSN 0090-5364. doi: 10.1214/aos/1176347257.
URL http://dx.doi.org/10.1214/aos/1176347257.
Michael Woodroofe. Integrable expansions for posterior distributions for one-parameter exponential families. Statist. Sinica, 2(1):91–111, 1992. ISSN 1017-0405.

20

