class_docstr: Wake-Sleep algorithm [@hinton1995wake].
  
  Given a probability model $p(x, z; \theta)$ and variational
  distribution $q(z\mid x; \lambda)$, wake-sleep alternates between
  two phases:
  
  + In the wake phase, $\log p(x, z; \theta)$ is maximized with
  respect to model parameters $\theta$ using bottom-up samples
  $z\sim q(z\mid x; \lambda)$.
  + In the sleep phase, $\log q(z\mid x; \lambda)$ is maximized with
  respect to variational parameters $\lambda$ using top-down
  "fantasy" samples $z\sim p(x, z; \theta)$.
  
  @hinton1995wake justify wake-sleep under the variational lower
  bound of the description length,
  
  $\mathbb{E}_{q(z\mid x; \lambda)} [
      \log p(x, z; \theta) - \log q(z\mid x; \lambda)].$
  
  Maximizing it with respect to $\theta$ corresponds to the wake phase.
  Instead of maximizing it with respect to $\lambda$ (which
  corresponds to minimizing $\text{KL}(q\|p)$), the sleep phase
  corresponds to minimizing the reverse KL $\text{KL}(p\|q)$ in
  expectation over the data distribution.
  
  #### Notes
  
  In conditional inference, we infer $z$ in $p(z, \beta
  \mid x)$ while fixing inference over $\beta$ using another
  distribution $q(\beta)$. During gradient calculation, instead
  of using the model's density
  
  $\log p(x, z^{(s)}), z^{(s)} \sim q(z; \lambda),$
  
  for each sample $s=1,\ldots,S$, `WakeSleep` uses
  
  $\log p(x, z^{(s)}, \beta^{(s)}),$
  
  where $z^{(s)} \sim q(z; \lambda)$ and $\beta^{(s)}
  \sim q(\beta)$.
  
  The objective function also adds to itself a summation over all
  tensors in the `REGULARIZATION_LOSSES` collection.
class_docstr_tok: ["Wake-Sleep","algorithm","[","@","hinton1995wake","]",".","Given","a","probability","model","$","p","(","x",",","z",";","\\theta",")","$","and","variational","distribution","$","q","(","z\\mid","x",";","\\lambda",")","$",",","wake-sleep","alternates","between","two","phases",":","+","In","the","wake","phase",",","$","\\log","p","(","x",",","z",";","\\theta",")","$","is","maximized","with","respect","to","model","parameters","$","\\theta","$","using","bottom-up","samples","$","z\\sim","q","(","z\\mid","x",";","\\lambda",")","$",".","+","In","the","sleep","phase",",","$","\\log","q","(","z\\mid","x",";","\\lambda",")","$","is","maximized","with","respect","to","variational","parameters","$","\\lambda","$","using","top-down","''","fantasy","''","samples","$","z\\sim","p","(","x",",","z",";","\\theta",")","$",".","@","hinton1995wake","justify","wake-sleep","under","the","variational","lower","bound","of","the","description","length",",","$","\\mathbb","{","E","}","_","{","q","(","z\\mid","x",";","\\lambda",")","}","[","\\log","p","(","x",",","z",";","\\theta",")","-","\\log","q","(","z\\mid","x",";","\\lambda",")","]",".","$","Maximizing","it","with","respect","to","$","\\theta","$","corresponds","to","the","wake","phase",".","Instead","of","maximizing","it","with","respect","to","$","\\lambda","$","(","which","corresponds","to","minimizing","$","\\text","{","KL","}","(","q\\|p",")","$",")",",","the","sleep","phase","corresponds","to","minimizing","the","reverse","KL","$","\\text","{","KL","}","(","p\\|q",")","$","in","expectation","over","the","data","distribution",".","#","#","#","#","Notes","In","conditional","inference",",","we","infer","$","z","$","in","$","p","(","z",",","\\beta","\\mid","x",")","$","while","fixing","inference","over","$","\\beta","$","using","another","distribution","$","q","(","\\beta",")","$",".","During","gradient","calculation",",","instead","of","using","the","model","'s","density","$","\\log","p","(","x",",","z^","{","(","s",")","}",")",",","z^","{","(","s",")","}","\\sim","q","(","z",";","\\lambda",")",",","$","for","each","sample","$","s=1",",","\\ldots",",","S","$",",","`","WakeSleep","`","uses","$","\\log","p","(","x",",","z^","{","(","s",")","}",",","\\beta^","{","(","s",")","}",")",",","$","where","$","z^","{","(","s",")","}","\\sim","q","(","z",";","\\lambda",")","$","and","$","\\beta^","{","(","s",")","}","\\sim","q","(","\\beta",")","$",".","The","objective","function","also","adds","to","itself","a","summation","over","all","tensors","in","the","`","REGULARIZATION_LOSSES","`","collection","."]
code: def __init__(self, *args, **kwargs):
      super(WakeSleep, self).__init__(*args, **kwargs)
  
file_path: edward/edward/inferences/wake_sleep.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import six
  import tensorflow
  from edward.inferences.variational_inference import VariationalInference
  from edward.models import RandomVariable
  from edward.util import copy, get_descendants
label_desc: Use image node_id 1 to create a new WakeSleep object from inherited base classes: VariationalInference with example: obj = WakeSleep()