class_docstr: Sampler for Bayesian regression with a spike-and-slab prior on weights.
  
  This implementation follows the sampler described in section 3.2
  of Scott and Varian, 2013 [1].
  
  ### Model
  
  This sampler assumes the regression model
  
  ```
  y ~ Normal(loc=matvec(design_matrix,  # `X` in [1].
                        weights),       # `beta` in `[1]`.
             scale=observation_noise_scale)  # `sigma_epsilon` in [1].
  ```
  
  where the design matrix has shape `[num_outputs, num_features]`, with a
  conjugate InverseGamma prior on the noise variance (eqn (6) of [1]):
  
  ```
  observation_noise_scale**2 ~ InverseGamma(
    concentration=observation_noise_variance_prior_concentration,
    scale=observation_noise_variance_prior_scale)
  ```
  
  and a spike-and-slab prior on the weights (eqns (5) and (6) of [1]):
  
  ```
  slab_weights ~ MultivariateNormal(
     loc=0.,  # `b` from [1].
     precision=(weights_prior_precision  # `Omega^{-1}` from [1].
                / observation_noise_scale**2))
  nonzeros ~ Bernoulli(probs=nonzero_prior_prob)  # `gamma` from [1].
  weights = slab_weights * nonzeros
  ```
  
  ### Example
  
  Constructing a sampler instance specifies the model priors:
  
  ```python
  sampler = spike_and_slab.SpikeSlabSampler(
    design_matrix=design_matrix,
    observation_noise_variance_prior_concentration=1.,
    observation_noise_variance_prior_scale=1.
    nonzero_prior_prob=0.1)
  ```
  
  The sampler instance itself is stateless, though some internal methods take
  or accept `SpikeSlabSamplerState` tuples representing posterior quantities
  maintained within a sampling pass. The sampler is
  invoked by passing the regression targets (`y`) and the initial sparsity
  pattern (`nonzeros`):
  
  ```
  (observation_noise_variance,
   weights) = sampler.sample_noise_variance_and_weights(
     targets=y, initial_nonzeros=tf.ones([num_features], dtype=tf.bool))
  ```
  
  This implements the stochastic search variable selection (SSVS) algorithm [2],
  sweeping over the features in random order to resample their sparsity
  indicators one by one. It then returns a sample from the joint posterior
  on the regression weights and the observation noise variance, conditioned
  on the resampled sparsity pattern.
  
  #### References
  
  [1] Steven L. Scott and Hal Varian. Predicting the Present with Bayesian
      Structural Time Series. __International Journal of Mathematical Modelling
      and Numerical Optimisation 5.1-2 (2014): 4-23.__
      https://people.ischool.berkeley.edu/~hal/Papers/2013/pred-present-with-bsts.pdf
  
  [2] George, E. I. and McCulloch, R. E. Approaches for Bayesian variable
      selection. __Statistica Sinica 7, 339–374 (1997)__.
class_docstr_tok: ["Sampler","for","Bayesian","regression","with","a","spike-and-slab","prior","on","weights",".","This","implementation","follows","the","sampler","described","in","section","3.2","of","Scott","and","Varian",",","2013","[","1","]",".","#","#","#","Model","This","sampler","assumes","the","regression","model","``","`","y","~","Normal","(","loc=matvec","(","design_matrix",",","#","`","X","`","in","[","1","]",".","weights",")",",","#","`","beta","`","in","`","[","1","]","`",".","scale=observation_noise_scale",")","#","`","sigma_epsilon","`","in","[","1","]",".","``","`","where","the","design","matrix","has","shape","`","[","num_outputs",",","num_features","]","`",",","with","a","conjugate","InverseGamma","prior","on","the","noise","variance","(","eqn","(","6",")","of","[","1","]",")",":","``","`","observation_noise_scale","*","*","2","~","InverseGamma","(","concentration=observation_noise_variance_prior_concentration",",","scale=observation_noise_variance_prior_scale",")","``","`","and","a","spike-and-slab","prior","on","the","weights","(","eqns","(","5",")","and","(","6",")","of","[","1","]",")",":","``","`","slab_weights","~","MultivariateNormal","(","loc=0.",",","#","`","b","`","from","[","1","]",".","precision=","(","weights_prior_precision","#","`","Omega^","{","-1","}","`","from","[","1","]",".","\/","observation_noise_scale","*","*","2",")",")","nonzeros","~","Bernoulli","(","probs=nonzero_prior_prob",")","#","`","gamma","`","from","[","1","]",".","weights","=","slab_weights","*","nonzeros","``","`","#","#","#","Example","Constructing","a","sampler","instance","specifies","the","model","priors",":","``","`","python","sampler","=","spike_and_slab.SpikeSlabSampler","(","design_matrix=design_matrix",",","observation_noise_variance_prior_concentration=1.",",","observation_noise_variance_prior_scale=1",".","nonzero_prior_prob=0.1",")","``","`","The","sampler","instance","itself","is","stateless",",","though","some","internal","methods","take","or","accept","`","SpikeSlabSamplerState","`","tuples","representing","posterior","quantities","maintained","within","a","sampling","pass",".","The","sampler","is","invoked","by","passing","the","regression","targets","(","`","y","`",")","and","the","initial","sparsity","pattern","(","`","nonzeros","`",")",":","``","`","(","observation_noise_variance",",","weights",")","=","sampler.sample_noise_variance_and_weights","(","targets=y",",","initial_nonzeros=tf.ones","(","[","num_features","]",",","dtype=tf.bool",")",")","``","`","This","implements","the","stochastic","search","variable","selection","(","SSVS",")","algorithm","[","2","]",",","sweeping","over","the","features","in","random","order","to","resample","their","sparsity","indicators","one","by","one",".","It","then","returns","a","sample","from","the","joint","posterior","on","the","regression","weights","and","the","observation","noise","variance",",","conditioned","on","the","resampled","sparsity","pattern",".","#","#","#","#","References","[","1","]","Steven","L.","Scott","and","Hal","Varian",".","Predicting","the","Present","with","Bayesian","Structural","Time","Series",".","__International","Journal","of","Mathematical","Modelling","and","Numerical","Optimisation","5.1-2","(","2014",")",":","4-23.__","https",":","\/\/people.ischool.berkeley.edu\/~hal\/Papers\/2013\/pred-present-with-bsts.pdf","[","2","]","George",",","E.","I.","and","McCulloch",",","R.","E.","Approaches","for","Bayesian","variable","selection",".","__Statistica","Sinica","7",",","339\u2013374","(","1997",")","__","."]
docstr: Proposes flipping the sparsity indicator of the `idx`th feature.
  
  This method computes the sampler state (including factorized precision
  matrices) for a given sparsity pattern, given the state for a
  related sparsity pattern that differs in a single position. This is
  achieved using rank-1 Cholesky updates running in time
  proportional to `num_features**2`, and so is typically more efficient than
  recomputing the equivalent state from scratch using
  `_initialize_sampler_state`.
  
  Args:
    sampler_state: instance of `SpikeSlabSamplerState` collecting (potentially
      batched) Tensor quantities relevant to the sampler. See the
      `SpikeSlabSamplerState` definition for details.
    idx: scalar int `Tensor` index in `[0, num_features)`. This is a single
      value shared across all batch elements.
  
  Returns:
    updated_sampler_state: instance of `SpikeSlabSamplerState` equivalent to
      `self._initialize_sampler_state(targets, new_nonzeros)`, where
      `new_nonzeros` is equal to `nonzeros` with the `idx`th entry
      negated.
code: def _flip_feature(self, sampler_state, idx):
      """Proposes flipping the sparsity indicator of the `idx`th feature.
  
      This method computes the sampler state (including factorized precision
      matrices) for a given sparsity pattern, given the state for a
      related sparsity pattern that differs in a single position. This is
      achieved using rank-1 Cholesky updates running in time
      proportional to `num_features**2`, and so is typically more efficient than
      recomputing the equivalent state from scratch using
      `_initialize_sampler_state`.
  
      Args:
        sampler_state: instance of `SpikeSlabSamplerState` collecting (potentially
          batched) Tensor quantities relevant to the sampler. See the
          `SpikeSlabSamplerState` definition for details.
        idx: scalar int `Tensor` index in `[0, num_features)`. This is a single
          value shared across all batch elements.
  
      Returns:
        updated_sampler_state: instance of `SpikeSlabSamplerState` equivalent to
          `self._initialize_sampler_state(targets, new_nonzeros)`, where
          `new_nonzeros` is equal to `nonzeros` with the `idx`th entry
          negated.
      """
      with tf.name_scope("flip_feature_indicator"):
          was_nonzero = tf.gather(sampler_state.nonzeros, idx, axis=-1)
          new_nonzeros = _set_vector_index(
              sampler_state.nonzeros, idx, tf.logical_not(was_nonzero)
          )
  
          # Update the weight posterior mean and precision for the new nonzeros.
          # (and also update the prior, used to compute the marginal likelihood).
          new_conditional_prior_precision_chol = _update_nonzero_block_chol(
              # Low-rank update equivalent to
              # `tf.linalg.cholesky(
              #    _select_nonzero_block(self.weights_prior_precision, nonzeros))`.
              chol=sampler_state.conditional_prior_precision_chol,
              idx=idx,
              psd_matrix=self.weights_prior_precision,
              new_nonzeros=new_nonzeros,
              previous_nonzeros=sampler_state.nonzeros,
          )
          new_conditional_posterior_precision_chol = _update_nonzero_block_chol(
              chol=sampler_state.conditional_posterior_precision_chol,
              idx=idx,
              psd_matrix=sampler_state.weights_posterior_precision,
              new_nonzeros=new_nonzeros,
              previous_nonzeros=sampler_state.nonzeros,
          )
          new_conditional_weights_mean = tf.where(
              new_nonzeros,
              tf.linalg.cholesky_solve(
                  new_conditional_posterior_precision_chol,
                  sampler_state.x_transpose_y[..., tf.newaxis],
              )[..., 0],
              0.0,
          )
          return self._compute_log_prob(
              nonzeros=new_nonzeros,
              conditional_prior_precision_chol=(
                  new_conditional_prior_precision_chol
              ),
              conditional_posterior_precision_chol=(
                  new_conditional_posterior_precision_chol
              ),
              conditional_weights_mean=new_conditional_weights_mean,
              weights_posterior_precision=sampler_state.weights_posterior_precision,
              observation_noise_variance_posterior_scale=(
                  sampler_state.observation_noise_variance_posterior_scale
                  - tf.reduce_sum(
                      (
                          new_conditional_weights_mean
                          - sampler_state.conditional_weights_mean
                      )
                      * sampler_state.x_transpose_y,
                      axis=-1,
                  )
                  / 2
              ),
              x_transpose_y=sampler_state.x_transpose_y,
          )
  
imports: import collections
  import tensorflow.compat.v2
  from tensorflow_probability.python.bijectors import softplus
  from tensorflow_probability.python.distributions import bernoulli
  from tensorflow_probability.python.distributions import inverse_gamma
  from tensorflow_probability.python.distributions import joint_distribution_auto_batched
  from tensorflow_probability.python.distributions import sample
  from tensorflow_probability.python.experimental.distributions.mvn_precision_factor_linop import MultivariateNormalPrecisionFactorLinearOperator
  from tensorflow_probability.python.internal import broadcast_util
  from tensorflow_probability.python.internal import dtype_util
  from tensorflow_probability.python.internal import parameter_properties
  from tensorflow_probability.python.internal import prefer_static
  from tensorflow_probability.python.internal import samplers
  from tensorflow_probability.python.internal import vectorization_util
  from tensorflow_probability.python.math import linalg
  from tensorflow_probability.python.mcmc.internal import util
label_desc: Use image node_id 4 for calling the SpikeSlabSampler obj's underlying member method code with example usage: obj._flip_feature(sampler_state, idx) and returns: self