class_docstr: Variational inference with the KL divergence
  
  $\text{KL}( p(z \mid x) \| q(z) ).$
  
  To perform the optimization, this class uses a technique from
  adaptive importance sampling [@oh1992adaptive].
  
  #### Notes
  
  `KLpq` also optimizes any model parameters $p(z\mid x;
  \theta)$. It does this by variational EM, maximizing
  
  $\mathbb{E}_{p(z \mid x; \lambda)} [ \log p(x, z; \theta) ]$
  
  with respect to $\theta$.
  
  In conditional inference, we infer $z` in $p(z, \beta
  \mid x)$ while fixing inference over $\beta$ using another
  distribution $q(\beta)$. During gradient calculation, instead
  of using the model's density
  
  $\log p(x, z^{(s)}), z^{(s)} \sim q(z; \lambda),$
  
  for each sample $s=1,\ldots,S$, `KLpq` uses
  
  $\log p(x, z^{(s)}, \beta^{(s)}),$
  
  where $z^{(s)} \sim q(z; \lambda)$ and$\beta^{(s)}
  \sim q(\beta)$.
  
  The objective function also adds to itself a summation over all
  tensors in the `REGULARIZATION_LOSSES` collection.
class_docstr_tok: ["Variational","inference","with","the","KL","divergence","$","\\text","{","KL","}","(","p","(","z","\\mid","x",")","\\|","q","(","z",")",")",".","$","To","perform","the","optimization",",","this","class","uses","a","technique","from","adaptive","importance","sampling","[","@","oh1992adaptive","]",".","#","#","#","#","Notes","`","KLpq","`","also","optimizes","any","model","parameters","$","p","(","z\\mid","x",";","\\theta",")","$",".","It","does","this","by","variational","EM",",","maximizing","$","\\mathbb","{","E","}","_","{","p","(","z","\\mid","x",";","\\lambda",")","}","[","\\log","p","(","x",",","z",";","\\theta",")","]","$","with","respect","to","$","\\theta","$",".","In","conditional","inference",",","we","infer","$","z","`","in","$","p","(","z",",","\\beta","\\mid","x",")","$","while","fixing","inference","over","$","\\beta","$","using","another","distribution","$","q","(","\\beta",")","$",".","During","gradient","calculation",",","instead","of","using","the","model","'s","density","$","\\log","p","(","x",",","z^","{","(","s",")","}",")",",","z^","{","(","s",")","}","\\sim","q","(","z",";","\\lambda",")",",","$","for","each","sample","$","s=1",",","\\ldots",",","S","$",",","`","KLpq","`","uses","$","\\log","p","(","x",",","z^","{","(","s",")","}",",","\\beta^","{","(","s",")","}",")",",","$","where","$","z^","{","(","s",")","}","\\sim","q","(","z",";","\\lambda",")","$","and","$","\\beta^","{","(","s",")","}","\\sim","q","(","\\beta",")","$",".","The","objective","function","also","adds","to","itself","a","summation","over","all","tensors","in","the","`","REGULARIZATION_LOSSES","`","collection","."]
docstr: Create an inference algorithm.
  
  Args:
    latent_vars: list of RandomVariable or
                 dict of RandomVariable to RandomVariable.
      Collection of random variables to perform inference on. If
      list, each random variable will be implictly optimized using a
      `Normal` random variable that is defined internally with a
      free parameter per location and scale and is initialized using
      standard normal draws. The random variables to approximate
      must be continuous.
docstr_tok: ["Create","an","inference","algorithm",".","Args",":","latent_vars",":","list","of","RandomVariable","or","dict","of","RandomVariable","to","RandomVariable",".","Collection","of","random","variables","to","perform","inference","on",".","If","list",",","each","random","variable","will","be","implictly","optimized","using","a","`","Normal","`","random","variable","that","is","defined","internally","with","a","free","parameter","per","location","and","scale","and","is","initialized","using","standard","normal","draws",".","The","random","variables","to","approximate","must","be","continuous","."]
code: def __init__(self, latent_vars=None, data=None):
      """Create an inference algorithm.
  
      Args:
        latent_vars: list of RandomVariable or
                     dict of RandomVariable to RandomVariable.
          Collection of random variables to perform inference on. If
          list, each random variable will be implictly optimized using a
          `Normal` random variable that is defined internally with a
          free parameter per location and scale and is initialized using
          standard normal draws. The random variables to approximate
          must be continuous.
      """
      if isinstance(latent_vars, list):
          with tf.variable_scope(None, default_name="posterior"):
              latent_vars_dict = {}
              continuous = (
                  "01",
                  "nonnegative",
                  "simplex",
                  "real",
                  "multivariate_real",
              )
              for z in latent_vars:
                  if (
                      not hasattr(z, "support")
                      or z.support not in continuous
                  ):
                      raise AttributeError(
                          "Random variable {} is not continuous or a random "
                          "variable with supported continuous support.".format(
                              z
                          )
                      )
                  batch_event_shape = z.batch_shape.concatenate(
                      z.event_shape
                  )
                  loc = tf.Variable(tf.random_normal(batch_event_shape))
                  scale = tf.nn.softplus(
                      tf.Variable(tf.random_normal(batch_event_shape))
                  )
                  latent_vars_dict[z] = Normal(loc=loc, scale=scale)
              latent_vars = latent_vars_dict
              del latent_vars_dict
  
      super(KLpq, self).__init__(latent_vars, data)
  
file_path: edward/edward/inferences/klpq.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import six
  import tensorflow
  from edward.inferences.variational_inference import VariationalInference
  from edward.models import RandomVariable
  from edward.util import copy, get_descendants
label_desc: Use image node_id 1 to create a new KLpq object from inherited base classes: VariationalInference with example: obj = KLpq(latent_vars, data)