Multiplying a Gaussian Matrix by a Gaussian Vector

arXiv:1702.02815v2 [math.PR] 5 Apr 2017

Pierre-Alexandre Mattei

pierre-alexandre.mattei@parisdescartes.fr

Laboratoire MAP5, UMR CNRS 8145
Université Paris Descartes & Sorbonne Paris Cité
45 rue des Saints-Pères, 75006 Paris, France

Abstract
We provide a new and simple characterization of the multivariate generalized Laplace
distribution. In particular, this result implies that the product of a Gaussian matrix with
independent and identically distributed columns by an independent isotropic Gaussian
vector follows a symmetric multivariate generalized Laplace distribution.
Keywords: Gamma distribution; Laplace distribution; Product distribution; Random
matrix; Scale mixture; Variance-gamma distribution

1. Introduction
Wishart and Bartlett (1932) proved that the inner product of two independent bidimensional
standard gaussian vectors follows a Laplace distribution. This result is deeply linked to the
fact that the Laplace distribution can be represented as an infinite scale mixture of Gaussians
with gamma mixing distribution. Specifically, if σ 2 follows a Gamma(1, 1/2) distribution
and x|σ ∼ N (0, σ 2 ), then x follows a standard Laplace distribution1 . This representation –
which was recently named the Gauss-Laplace representation by Ding and Blitzstein (2017)
following a blog post by Christian P. Robert2 – is particularly useful if one wants to simulate
a Laplace random variable: its use constitutes for example the cornerstone of the Gibbs
sampling scheme for the Bayesian lasso of Park and Casella (2008).
In this short paper, we are interested in studying links between multivariate counterparts
of these two characterizations. More specifically, we give a new simple characterization of the
multivariate generalized Laplace distribution of Kotz, Kozubowski, and Podgórski (2001).
As a corollary, we show that the product of a zero-mean Gaussian matrix with in dependent and identically distributed (i.i.d.) columns and a zero-mean isotropic Gaussian vector
follows a symmetric multivariate generalized Laplace distribution, a result that has useful
applications for Bayesian principal component analysis (Bouveyron, Latouche, and Mattei,
2016, 2017).
In the remainder of this paper, p and d are two positive integers and Sp+ denotes the
cone of positive semidefinite matrices of size p × p.

2. The multivariate generalized Laplace distribution
While the definition of the univariate Laplace distribution is widely undisputed, there exist
several different generalizations of this distribution to higher dimensions – a comprehensive
review of such generalizations can be found in the monograph of Kotz, Kozubowski, and Podgórski
1. The shape-rate parametrization of the gamma distribution is used through this paper. Note also that a
standard Laplace distribution is centered with variance two.
2. https://xianblog.wordpress.com/2015/10/14/gauss-to-laplace-transmutation/

1

(2001). In particular, McGraw and Wagner (1968) introduced a zero-mean elliptically contoured bidimensional Laplace distribution with univariate Laplace marginals. This distribution was later generalized to the p-dimensional setting by Anderson (1992), considering
characteristic functions of the form
1
∀u ∈ Rp , φ(u) =
,
1 T
1 + 2 u Σu
where Σ ∈ Sp+ . This distribution was notably promoted by Eltoft, Kim, and Lee (2006)
and is arguably the most popular multivariate generalization of the Laplace distribution
(Kotz, Kozubowski, and Podgórski, 2001, p. 229). Among its advantages, this distribution
can be slightly generalized to model skewness, by building on characteristic functions of the
form
1
,
∀u ∈ Rp , φ(u) =
1 T
1 + 2 u Σu − iµT u

where µ ∈ Rp accounts for asymmetry. Similarly to the univariate Laplace distribution, this
asymmetric multivariate generalization is infinitely divisible (Kotz, Kozubowski, and Podgórski,
2001, p. 256). Therefore, it can be associated with a specific Lévy process (Kyprianou,
2014, p. 5), whose increments will follow yet another generalization of the Laplace distribution, the multivariate generalized asymmetric Laplace distribution. This distribution,
introduced by Kotz, Kozubowski, and Podgórski (2001, p. 257) and further studied by
Kozubowski, Podgórski, and Rychlik (2013), will be the cornerstone of our analysis of multivariate caracterizations of Laplace and Gaussian distributions.

Definition 1 A random variable z ∈ Rp is said to have a multivariate generalized
asymmetric Laplace distribution with parameters s > 0, µ ∈ Rp and Σ ∈ Sp+ if its
characteristic function is
!s
1
p
.
∀u ∈ R , φGALp (Σ,µ,s) (u) =
1 + 12 uT Σu − iµT u
It is denoted by z ∼ GALp (Σ, µ, s).
General properties of the generalized asymmetric Laplace distribution distribution are discussed by Kozubowski, Podgórski, and Rychlik (2013). We list here a few useful ones.
Proposition 2 Let s > 0, µ ∈ Rp and Σ ∈ Sp+ . If z ∼ GALp (Σ, µ, s), we have E(z) = sµ
and Cov(z) = s(Σ + µµT ). Moreover, if Σ is positive definite, the density of z is given by


T −1
2eµ Σ x
QΣ (x) s−p/2
p
√
∀x ∈ R , fz (x) =
Ks−p/2 (QΣ (x)C(Σ, µ)) ,
(2π)p/2 Γ(s) det Σ C(Σ, µ)
√
p
where QΣ (x) = xT Σ−1 x, C(Σ, µ) = 2 + µT Σ−1 µ and Ks−p/2 is the modified Bessel
function of the second kind of order s − p/2.
Note that the GAL1 (2b2 , 0, 1) case corresponds to a centered univariate Laplace distribution
with scale parameter b > 0. In the symmetric case (µ = 0) and when s = 1, we recover the
multivariate generalization of the Laplace distribution of Anderson (1992).
An appealing property of the multivariate generalized Laplace distribution is that it is
also endowed with a multivariate counterpart of the Gauss-Laplace representation.
2

Theorem 3 (Generalized Gauss-Laplace representation) Let s > 0, µ ∈ Rp and Σ ∈
Sp+ . If u ∼ Gamma(s, 1) and x ∼ N (0, Σ) is independent of u, we have
√
uµ + ux ∼ GALp (Σ, µ, s).
(1)
A proof of this result can be found in Kotz, Kozubowski, and Podgórski (2001, chap. 6).
This representation explains why the multivariate generalized Laplace distribution can also
be seen as a multivariate generalization of the variance-gamma distribution which is widely
used in the field of quantitative finance (Madan, Carr, and Chang, 1998). Infinite mixtures
similar to (1) are called variance-mean mixtures (Barndorff-Nielsen, Kent, and Sørensen,
1982) and are discussed for example by Yu (2017).
Another useful property of the multivariate generalized Laplace distribution is that,
under some conditions, it is closed under convolution.
Proposition 4 Let s1 , s2 > 0, µ ∈ Rp and Σ ∈ Sp+ . If z1 ∼ GALp (Σ, µ, s1 ) and z2 ∼
GALp (Σ, µ, s2 ) are independant random variables, then
z1 + z2 ∼ GALp (Σ, µ, s1 + s2 ).

(2)

Proof Since z1 and z2 are independent, we have for all u ∈ Rp ,
φz1 +z2 (u) = φGALp (Σ,µ,s1 ) (u)φGALp (Σ,µ,s2 ) (u) =

1
1 T
1 + 2 u Σu − iµT u

!s1 +s2

which is the characteristic function of the GALp (Σ, µ, s1 + s2 ) distribution.

3. A new characterization involving a product between a Gaussian
matrix and a Gaussian vector
We now state our main theorem, which gives a new characterization of multivariate generalized Laplace distributions with half-integer shape parameters.
Theorem 5 Let W be a p × d random matrix with i.i.d. columns following a N (0, Σ)
distribution, y ∼ N (0, Id ) be a Gaussian vector independent from W and let µ ∈ Rp . We
have
Wy + ||y||22 µ ∼ GALp (2Σ, 2µ, d/2).
(3)
Proof For each k ∈ {1, ..., d} let wk be the k-th column of W, uk = yk2 and ξ k = yk wk +yk2 µ.
To prove the theorem, we will prove that ξ 1 , ..., ξ d follow a GAL distribution and use the
decomposition
d
X
ξk .
Wy + ||y||22 µ =
k=1

Let k ∈ {1, ..., d}. Since y is standard Gaussian, uk = yk2 follows a χ2 (1) distribution, or equivalently a Gamma(1/2, 1/2) distribution. Therefore, uk /2 ∼ Gamma(1/2, 1).
3

Moreover, note that

√

d

uk wk = |yk |wk = yk sign(yk )wk = yk wk since |yk | and sign(yk ) are
d

independent and sign(yk )wk = wk . Therefore, according to the generalized Gauss-Laplace
representation, we have
r
uk √
uk
d
ξk =
2wk + 2µ ∼ GALp (2Σ, 2µ, 1/2).
2
2

Since ξ 1 , ..., ξ d are i.i.d. and following a GALp (2Σ, 2µ, 1/2) distribution, we can use
Proposition 4 to conclude that
Wy + ||y||22 µ =

d
X
k=1

ξ k ∼ GALp (2Σ, 2µ, d/2).

In the symmetric case (µ = 0), this result gives the distribution of the product between
a Gaussian matrix with i.i.d. columns and a isotropic Gaussian vector.
Corollary 6 Let W be a p × d random matrix with i.i.d. columns following a N (0, Σ)
distribution and let y ∼ N (0, αId ) be a Gaussian vector independent from W. Then
Wy ∼ GALp (2αΣ, 0, d/2).

(4)

Moreover, if u is a standard Gamma variable with shape parameter d/2 and if x ∼ N (0, 2αΣ)
is a Gaussian vector independent of u, then
d

Wy =

√

ux.

(5)

Less general versions of Theorem 5 have been proven in the past, dating back to the
derivation of the inner product of two i.i.d. standard Gaussian vectors by Wishart and Bartlett
(1932). In particular, the unidimensional case (p = 1) was recently proven by Gaunt (2014)
in order to obtain bounds for the convergence rate of random sums involving Gaussian
products. The multivariate symmetric isotropic case (µ = 0 and Σ proportional to Ip ) was
proven by Bouveyron, Latouche, and Mattei (2016) in order to derive the marginal likelihood of the noiseless probabilistic principal component analysis model of Roweis (1998).
While the proof of Bouveyron, Latouche, and Mattei (2016) relied on characteristic functions and the properties of Bessel functions, the proof that we presented here is closer in
spirit to the one of Gaunt (2014), based on representations of distributions.

4. Perspectives
The new characterization presented in this paper may notably prove useful in two contexts.
First, it indicates a new way of handling situations involving the product of a Gaussian matrix and a Gaussian vector. An important instance is the Bayesian factor analysis
model (Lopes and West, 2004), of which principal component analysis is a particular case.
4

In this framework, the marginal distribution of the data, which is essential for model selection purposes, can be derived using representation (5) together with the Gauss-Laplace
representation (Bouveyron, Latouche, and Mattei, 2016, 2017).
Moreover, our characterization offers a means to get around problems encountered when
dealing with distributions related to the GAL distribution. For example, representation
(3) might lead to alternative estimation strategies for some problems related to portfolio allocation (Mencía and Sentana, 2009; Breymann and Lüthi, 2013) or cluster analysis
(McNicholas, McNicholas, and Browne, 2013; Franczak, Browne, and McNicholas, 2014).

Acknowledgements
I thank Charles Bouveyron, Pierre Latouche, Brendan Murphy and Christian Robert for
fruitful advices and discussions. Part of this work was made during a visit to University
College Dublin, funded by the Fondation Sciences Mathématiques de Paris (FSMP).

References
D. N. Anderson. A multivariate Linnik distribution. Statistics & Probability Letters, 14(4):
333–336, 1992.
O. Barndorff-Nielsen, J. Kent, and M. Sørensen. Normal variance-mean mixtures and z
distributions. International Statistical Review/Revue Internationale de Statistique, 50(2):
145–159, 1982.
C. Bouveyron, P. Latouche, and P.-A. Mattei. Bayesian variable selection for globally sparse
probabilistic PCA. Technical report, HAL-01310409, Université Paris Descartes, 2016.
C. Bouveyron, P. Latouche, and P.-A. Mattei. Exact dimensionality selection for Bayesian
PCA. Technical report, HAL-01484099, Université Paris Descartes, 2017.
W. Breymann and D. Lüthi. ghyp: A package on generalized hyperbolic distributions.
Manual for R Package ghyp, 2013.
P. Ding and J. K. Blitzstein. On the Gaussian mixture representation of the Laplace distribution. The American Statistician, in press, 2017.
T. Eltoft, T. Kim, and T.-W. Lee. On the multivariate Laplace distribution. IEEE Signal
Processing Letters, 13(5):300–303, 2006.
B. C. Franczak, R. P. Browne, and P. D. McNicholas. Mixtures of shifted asymmetric
Laplace distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence,
36(6):1149–1157, 2014.
R. E. Gaunt. Variance-Gamma approximation via Stein’s method. Electronic Journal of
Probability, 19(38):1–33, 2014.
S. Kotz, T. Kozubowski, and K. Podgórski. The Laplace distribution and generalizations: a
revisit with applications to communications, exonomics, engineering, and finance. Springer
Science & Business Media, 2001.
5

T. Kozubowski, K. Podgórski, and I. Rychlik. Multivariate generalized Laplace distribution
and related random fields. Journal of Multivariate Analysis, 113:59–72, 2013.
A. Kyprianou. Fluctuations of Lévy processes with applications: Introductory Lectures.
Springer Science & Business Media, 2014.
H. F. Lopes and M. West. Bayesian model assessment in factor analysis. Statistica Sinica,
pages 41–67, 2004.
D. B. Madan, P. P. Carr, and E. C. Chang. The variance gamma process and option pricing.
European Finance Review, 2(1):79–105, 1998.
D. McGraw and J. Wagner. Elliptically symmetric distributions. IEEE Transactions on
Information Theory, 14(1):110–120, 1968.
S. M. McNicholas, P. D. McNicholas, and R. P. Browne. Mixtures of variance-gamma
distributions. arXiv preprint arXiv:1309.2695, 2013.
J. Mencía and E. Sentana. Multivariate location–scale mixtures of normals and mean–
variance–skewness portfolio allocation. Journal of Econometrics, 153(2):105–121, 2009.
T. Park and G. Casella. The Bayesian lasso. Journal of the American Statistical Association,
103(482):681–686, 2008.
S. Roweis. EM algorithms for PCA and SPCA. Advances in Neural Information Processing
Systems, 10:626–632, 1998.
J. Wishart and M. S. Bartlett. The distribution of second order moment statistics in a
normal system. Mathematical Proceedings of the Cambridge Philosophical Society, 28(4):
455–459, 1932.
Y. Yu. On normal variance–mean mixtures. Statistics & Probability Letters, 121:45–50,
2017.

6

