class_docstr: Convergence criterion based on inner products between successive gradients.
  
  Let `g[t]` be the gradient vector at step `t`, and `g[t-1]` the previous
  gradient. Their inner product:
  
  ```python
  grad_inner_product[t] = sum_i(g[t, i] * g[t - 1, i])
  ```
  
  measures correlation between successive optimization steps.
  We expect this to be positive if the optimization is making progress;
  conversely, it can be shown to be negative in expectation at the stationary
  distribution of constant-step-size SGD [(Pflug, 1990)][2].
  
  This criterion detects convergence when an exponentially-weighted moving
  average of `grad_inner_product` becomes negative; intuitively, when there has
  been no consistent direction to the most recent `window_size` steps.
  
  Theoretical analysis shows that with no decay (`window_size=np.inf`), this
  rule stops in finite time almost surely, for constant-step-size SGD under
  standard assumptions ([Pflug, 1990][2]; [Chee and Toulis, 2017][1]). In
  practice, it is often more efficient to use a decaying moving average.
  
  **Batch semantics**: because this criterion does not depend on the loss,
  vector-valued losses will not produce vector-valued convergence indicators.
  Instead, the returned `has_converged` is always scalar, and is computed from
  the inner product summed across gradients from all variables being
  optimized.
  
  TODO(b/150151919): If per-batch convergence indicators are important to you,
  please contact `tfprobability@tensorflow.org`.
  
  #### References
  
  [1] Jerry Chee and Panos Toulis. Convergence diagnostics for stochastic
  gradient descent with constant step size. _arXiv preprint arXiv:1710.06382,
  2017. https://arxiv.org/abs/1710.06382
  
  [2] Georg Ch. Pflug. Non-asymptotic confidence bounds for stochastic
  approximation algorithms with constant step size. Monatshefte fur Mathematik,
  110(3-4), pp.297-314, 1990.
code: def _bootstrap(self, loss, grads, parameters):
      initial_moving_product = tf.zeros(
          tf.shape(self.window_size), dtype=loss.dtype
      )
      return (grads, initial_moving_product)
  
imports: import tensorflow.compat.v2
  from tensorflow_probability.python.optimizer.convergence_criteria import convergence_criterion
label_desc: Use image node_id 3 for calling the SuccessiveGradientsAreUncorrelated obj's underlying member method code with example usage: obj._bootstrap(loss, grads, parameters) and returns: grads,  initial_moving_product