# Specifying a Laplace prior using a Gaussian random variable with Gamma variance

I need to place a Laplace prior on a random variable, however, I want to use a Gaussian distribution whose variance is Gamma(1,1) distributed, i.e.,

\begin{align} x &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

My question is: Given this above setup, what is the corresponding form of my Laplace distribution (location and scale) for this Laplace prior?

• I'm trying to clarify. You have X~normal(0,y) where y~gamma(1,1)? Where does the Laplace come into play? – Eric Sep 4 '14 at 14:24
• Is this code from infer.net? I think what you are doing is modelling that your variable $x$ is a zero mean and you have a gamma prior on the variance. This is usually done because of conjugate properties. I have no idea where the Laplace distribution comes from. I think you need to edit your question to make it more clear. – Luca Sep 4 '14 at 15:48
• Yes exactly this is infer.net. Actually I read this post: social.microsoft.com/Forums/en-US/… I'm trying to have a Laplace prior. but I need to specify the parameters for this Laplace prior. – user3034939 Sep 4 '14 at 15:51
• I am not sure. I know you can generate a Laplace distribution from subtraction of exponential distributions. Perhaps that is what he was hinting at? – Luca Sep 4 '14 at 16:30
• I edited for clarity. So basically if you have a normal random variable whose variance comes from a Gamma(1,1) distribution then that random variable will approximate a Laplace distribution. What the OP is asking is how to determine the parameters of the corresponding Laplace distribution. – Dan Sep 4 '14 at 17:27

So the question seems to have stemmed from a discussion on this forum: http://social.microsoft.com/Forums/en-US/47b613d0-177d-4ce3-b54d-2476144ece6b/double-exponential-prior-migrated-from-communityresearchmicrosoftcom?forum=infer.net

The general idea, that the forum alludes to, is that you can build a distribution that resembles a Laplace distribution from a Normal distribution by specifying the variance of the Normal distribution to follow a Gamma distribution, i.e.,

\begin{align} x|\sigma^2 &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

However, as the OP asks, how to specify the location ($\mu$) and scale ($b$) of the corresponding Laplace distribution is not obvious. And in fact, you can't since the distribution will not be exactly a Laplace distribution, but just something very similar.

To better understand the question we (below) a plot of many Laplace distibutions with differing locations ($\mu$) and scales ($b$).

And so generating random variables from Normal distribution with variance following a Gamma(1,1) distribution will give us something that looks very close to a Laplace distribution.

However, it is only an approximation and is not exact.

On the other hand, there is a way to generate a Laplace distribution (with known location ($\mu$) and scale ($b$)) by using a Normal distribution with variance corresponding to a Rayleigh distribution.

The result is the following:

If $X|Y \sim N(\mu,\sigma=Y)$ with $Y \sim \text{Rayleigh}(b)$ then $X \sim \text{Laplace}(\mu, b)$.

• There is a link between the Rayleigh distribution and the Gamma distribution that may get you close to what you are looking for as well. – Dan Sep 4 '14 at 17:53
• Thanks for your nice explanation :) I think I cannot use this similarity to reach my goal. I should think about that. – user3034939 Sep 5 '14 at 17:00
• I found this paper. Maybe useful for this discussion: Sparse probabilistic projections - by: C´edric Archambeau, Francis R. Bach – user3034939 Oct 3 '14 at 14:42
• This answer is incorrect. Generating random variables from Normal distribution with variance following a Gamma(1,1) distribution gives exactly a Laplace distribution, not an approximation. This follows immediately from your Rayleigh construction since the square of Rayleigh is Gamma with shape 1. – Tom Minka Oct 3 '14 at 15:35
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 2, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.9449149370193481, "perplexity": 526.3065927555098}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2021-04/segments/1610703529128.47/warc/CC-MAIN-20210122051338-20210122081338-00438.warc.gz"}