# Gibbs Sampling

Gibbs sampling is a special case of the Metropolis-Hastings method, where a sequence of proposal distributions $$q$$ is defined in terms of the conditional distributions of the joint distribution $$p(\mathbf{x})$$, and proposals are always accepted.

In the general case of a system with $$K$$ variables, a single iteration involves sampling one parameter at a time:

\begin{equation} \begin{array}{l}{x_{1}^{(t+1)} \sim P\left(x_{1} | x_{2}^{(t)}, x_{3}^{(t)}, \ldots, x_{K}^{(t)}\right)} \ {x_{2}^{(t+1)} \sim P\left(x_{2} | x_{1}^{(t+1)}, x_{3}^{(t)}, \ldots, x_{K}^{(t)}\right)} \ {x_{3}^{(t+1)} \sim P\left(x_{3} | x_{1}^{(t+1)}, x_{2}^{(t+1)}, \ldots, x_{K}^{(t)}\right), \text { etc. }}\end{array} \end{equation}

## Pros and Cons

1. Suffers the same defects as Metropolis-Hastings methods
{"ft_lang_label":"__label__en","ft_lang_prob":0.7218134,"math_prob":0.9999931,"size":859,"snap":"2021-43-2021-49","text_gpt3_token_len":306,"char_repetition_ratio":0.12163743,"word_repetition_ratio":0.0,"special_character_ratio":0.3760186,"punctuation_ratio":0.09883721,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9999831,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2021-11-27T06:22:07Z\",\"WARC-Record-ID\":\"<urn:uuid:3774b11d-4c3a-45ac-bbf7-6ea7727ffe76>\",\"Content-Length\":\"1049005\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:19932d98-39c3-440b-b1dc-e12f617cf0ba>\",\"WARC-Concurrent-To\":\"<urn:uuid:7906f691-e1af-4b69-bccd-5bdca532ea95>\",\"WARC-IP-Address\":\"68.183.27.198\",\"WARC-Target-URI\":\"https://braindump.jethro.dev/posts/gibbs_sampling/\",\"WARC-Payload-Digest\":\"sha1:HBVYI74TUYGF44ZJMK4MO6KKOYSQDHMS\",\"WARC-Block-Digest\":\"sha1:XSZMR6B3F3BG4ND2JUX7TYI5SI266XDI\",\"WARC-Truncated\":\"length\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2021/CC-MAIN-2021-49/CC-MAIN-2021-49_segments_1637964358118.13_warc_CC-MAIN-20211127043716-20211127073716-00022.warc.gz\"}"}