code: # Copyright 2016 The TensorFlow Authors. All Rights Reserved.
  #
  # Licensed under the Apache License, Version 2.0 (the "License");
  # you may not use this file except in compliance with the License.
  # You may obtain a copy of the License at
  #
  #     http://www.apache.org/licenses/LICENSE-2.0
  #
  # Unless required by applicable law or agreed to in writing, software
  # distributed under the License is distributed on an "AS IS" BASIS,
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  # See the License for the specific language governing permissions and
  # limitations under the License.
  # ==============================================================================
  """Framework of debug wrapper sessions.
  
  A debug wrapper session is a wrapper around a TensorFlow Python Session.
  The wrapper preserves the Session interface, most importantly the run() method,
  while providing abilities to:
  a) Intercept a run() call to a wrapped session and insert debug tensor watches
     according to externally-specified debug URLs.
  
  b) Release control to an external (i.e., non-Session) object before and after
     the run() call, so that the external object can perform actions such as
     launching a UI to let users inspect the intermediate tensors and partition
     graphs from the run() call.
  
  c) (To be implemented) Intercept a run() call and give control to DebugStepper
     to let it perform stepping / continuing-to actions on the graph.
  
  b) (To be implemented in a future CL) Enter an instruction loop to let an
     external object (e.g., remote client) launch run() and cont() calls
     remotely.
  
  *** The lifetime of a debug wrapper session: ***
  
  1) The wrapper session is created by calling the constructor with a
     wrapped (normal) session as the argument:
       wrapper = FooDebugWrapperSession(sess)
     wherein FooDebugWrapperSession is a concrete subclass implementing the
     abstract BaseDebugWrapperSession class below.
  
  2) Near the end of the constructor call, the on_session_init() callback is
     invoked, with a OnSessionInitRequest object as the argument. The object
     carries the wrapped (normal) session object.
  
  3) The callback handles the request and returns a OnSessionInitResponse
     object with an action field, directing the wrapper session what to do next.
  
  If the action field in the OnSessionInitResponse is PROCEED, the constuctor
  returns. Control is released back to the caller of the constructor, which can
  invoke run() method of wrapper session with the same syntax as a non-wrapped
  session, e.g.,:
    wrapper.run(fetches, feed_dict=feeds, options=run_options)
  
  Below, A1 - A2 is the lifetime of a wrapper run() call if the action is
  PROCEED:
  
  A1) Right at the start of each run() call, the on_run_start() callback is
      invoked, with an OnRunStartRequest object carrying information such as
      the fetches, the feed dict, the run options and run metadata used in
      this run call, along with a count of how many run calls has occurred
      on this wrapper session. The callback then returns an OnRunStartResponse
      object, of which the action field directs what the wrapper session
      actually will do of the run() call.
  
      If the action is DEBUG_RUN, a debugged (tensor-watched) run will ensue,
      with the debug URLs supplied in the debug_urls field of the response.
      These can be file:// or grpc:// URLs, for example.
  
      If the action is NON_DEBUG_RUN, a non-debug (normal) run will ensue.
  
      If the action is INVOKE_STEPPER, no run() call will be issued to the
      wrapped session. But instead, a DebugStepper (i.e., "continuation
      debugger") will be used to perform stepping / continue-to actions on
      the graph.
  
  TODO(cais): The event loop for the DebugStepper will request additional
     callbacks including on_cont_start() and on_cont_end(). Add those.
  
  A2) Right before the run() returns, the on_run_end() callback is invoked,
      with an OnRunEndRequest object as the argument, which carries information
      including the actual action performed in the warpper run() call and the
      run_metadata from the run() call.
  
  However, if the action field in OnSessionInitResponse is
  REMOTE_INSTR_LOOP, the constructor will automatically invoke an instruction loop
  that gives the control to a remote caller.
  
  In the remote instruction loop, the following steps will happen:
  
  B1) Callback on_instr_start() is invoked. The callback will return an
      OnInstrStartResponse object with an action field which can order one of
      the following actions:
          i) a run() call with fetches, feeds and debug_urls specified.
         ii) a DebugStepper cont() call with target specified.
        iii) value overrides in the cached tensors from the DebugStepper.
         iv) exit the instruction loop.
  
  B2) The wrapper session carries out the action specified above.
  
  B3) If still in the instruction loop, the wrapper session invokes the
      on_instr_end() callback. After the on_instr_end() callback returns, jump
      back to B1.
  
  TODO(cais): Implemented the instruction loop in B1 - B3.
  
  """
  
  from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  
  import abc
  
  from tensorflow.core.protobuf import config_pb2
  from tensorflow.python.client import session
  from tensorflow.python.debug import debug_utils
  from tensorflow.python.debug import stepper
  from tensorflow.python.framework import errors
  
  
  # Helper function.
  def _check_type(obj, expected_type):
    """Check if an object is of the expected type.
  
    Args:
      obj: The object being checked.
      expected_type: (type) The expected type of obj.
  
    Raises:
        TypeError: If obj is not an instance of expected_type.
    """
    if not isinstance(obj, expected_type):
      raise TypeError("Expected type %s; got type %s" %
                      (expected_type, type(obj)))
  
  
  class OnSessionInitRequest(object):
    """Request to an on-session-init callback.
  
    This callback is invoked during the __init__ call to a debug-wrapper session.
    """
  
    def __init__(self, sess):
      """Constructor.
  
      Args:
        sess: A tensorflow Session object.
      """
  
      _check_type(sess, session.BaseSession)
      self.session = sess
  
  
  class OnSessionInitAction(object):
    """Enum-like values for possible action to take on session init."""
  
    # Proceed, without special actions, in the wrapper session initialization.
    # What action the wrapper session performs next is determined by the caller
    # of the wrapper session. E.g., it can call run().
    PROCEED = "proceed"
  
    # Instead of letting the caller of the wrapper session determine what actions
    # the wrapper session will perform next, enter a loop to receive instructions
    # from a remote client.
    # For example, TensorBoard visual debugger can use this action so that it can
    # launch session.run() calls remotely.
    REMOTE_INSTR_LOOP = "remote_instr_loop"
  
  
  class OnSessionInitResponse(object):
    """Response from an on-session-init callback."""
  
    def __init__(self, action):
      """Constructor.
  
      Args:
        action: (`OnSessionInitAction`) Debugger action to take on session init.
      """
      _check_type(action, str)
      self.action = action
  
  
  class OnRunStartRequest(object):
    """Request to an on-run-start callback.
  
    This callback is invoked during a run() call of the debug-wrapper
    session, immediately after the run() call counter is incremented.
    """
  
    def __init__(self, fetches, feed_dict, run_options, run_metadata,
                 run_call_count):
      """Constructor of `OnRunStartRequest`.
  
      Args:
        fetches: Fetch targets of the run() call.
        feed_dict: The feed dictionary to the run() call.
        run_options: RunOptions input to the run() call.
        run_metadata: RunMetadata input to the run() call.
          The above four arguments are identical to the input arguments to the
          run() method of a non-wrapped TensorFlow session.
        run_call_count: 1-based count of how many run calls (including this one)
          has been invoked.
      """
      self.fetches = fetches
      self.feed_dict = feed_dict
      self.run_options = run_options
      self.run_metadata = run_metadata
      self.run_call_count = run_call_count
  
  
  class OnRunStartAction(object):
    """Enum-like values for possible action to take on start of a run() call."""
  
    # Run once with debug tensor-watching.
    DEBUG_RUN = "debug_run"
  
    # Run without debug tensor-watching.
    NON_DEBUG_RUN = "non_debug_run"
  
    # Instead of running the fetches as a whole, as would normally happen, invoke
    # the (to-be-implemented) debug stepper.
    # TODO(cais): Remove "to-be-implemented".
    INVOKE_STEPPER = "invoke_stepper"
  
  
  class OnRunStartResponse(object):
    """Request from an on-run-start callback.
  
    The caller of the callback can use this response object to specify what
    action the debug-wrapper session actually takes on the run() call.
    """
  
    def __init__(self, action, debug_urls):
      """Constructor of `OnRunStartResponse`.
  
      Args:
        action: (`OnRunStartAction`) the action actually taken by the wrapped
          session for the run() call.
        debug_urls: (list of str) debug_urls used in watching the tensors during
          the run() call.
      """
  
      _check_type(action, str)
      self.action = action
  
      _check_type(debug_urls, list)
      self.debug_urls = debug_urls
  
  
  class OnRunEndRequest(object):
    """Request to an on-run-end callback.
  
    The callback is invoked immediately before the wrapped run() call ends.
    """
  
    def __init__(self,
                 performed_action,
                 run_metadata=None,
                 client_graph_def=None,
                 tf_error=None):
      """Constructor for `OnRunEndRequest`.
  
      Args:
        performed_action: (`OnRunStartAction`) Actually-performed action by the
          debug-wrapper session.
        run_metadata: run_metadata output from the run() call (if any).
        client_graph_def: (GraphDef) GraphDef from the client side, i.e., from
          the python front end of TensorFlow. Can be obtained with
          session.graph.as_graph_def().
        tf_error: (errors.OpError subtypes) TensorFlow OpError that occurred
          during the run (if any).
      """
  
      _check_type(performed_action, str)
      self.performed_action = performed_action
  
      if run_metadata is not None:
        _check_type(run_metadata, config_pb2.RunMetadata)
      self.run_metadata = run_metadata
      self.client_graph_def = client_graph_def
      self.tf_error = tf_error
  
  
  class OnRunEndResponse(object):
    """Response from an on-run-end callback."""
  
    def __init__(self):
  
      # Currently only a placeholder.
      pass
  
  
  class BaseDebugWrapperSession(session.SessionInterface):
    """Base class of debug-wrapper session classes.
  
    Concrete classes that inherit from this class need to implement the abstract
    methods such as on_session_init, on_run_start and on_run_end.
    """
  
    # TODO(cais): Add on_cont_start and on_cont_end callbacks once the stepper is
    # is available.
  
    def __init__(self, sess):
      """Constructor of `BaseDebugWrapperSession`.
  
      Args:
        sess: An (unwrapped) TensorFlow session instance.
  
      Raises:
        ValueError: On invalid `OnSessionInitAction` value.
      """
  
      _check_type(sess, session.BaseSession)
  
      # The session being wrapped.
      self._sess = sess
  
      # Keeps track of number of run calls that have been performed on this
      # debug-wrapper session.
      self._run_call_count = 0
  
      # Invoke on-session-init callback.
      response = self.on_session_init(OnSessionInitRequest(self._sess))
      _check_type(response, OnSessionInitResponse)
  
      if response.action == OnSessionInitAction.PROCEED:
        pass
      elif response.action == OnSessionInitAction.REMOTE_INSTR_LOOP:
        # TODO(cais): Implement REMOTE_INSTR_LOOP
        raise NotImplementedError(
            "OnSessionInitAction REMOTE_INSTR_LOOP has not been "
            "implemented.")
      else:
        raise ValueError(
            "Invalid OnSessionInitAction value: %s" % response.action)
  
    @property
    def graph(self):
      return self._sess.graph
  
    @property
    def sess_str(self):
      return self._sess.sess_str
  
    @property
    def session(self):
      return self._sess
  
    def run(self, fetches, feed_dict=None, options=None, run_metadata=None):
      """Wrapper around Session.run() that inserts tensor watch options.
  
      Args:
        fetches: Same as the `fetches` arg to regular `Session.run()`.
        feed_dict: Same as the `feed_dict` arg to regular `Session.run()`.
        options: Same as the `options` arg to regular `Session.run()`.
        run_metadata: Same as the `run_metadata` arg to regular `Session.run()`.
  
      Returns:
        Simply forwards the output of the wrapped `Session.run()` call.
  
      Raises:
        ValueError: On invalid `OnRunStartAction` value.
      """
  
      self._run_call_count += 1
  
      # Invoke on-run-start callback and obtain response.
      run_start_resp = self.on_run_start(
          OnRunStartRequest(fetches, feed_dict, options, run_metadata,
                            self._run_call_count))
      _check_type(run_start_resp, OnRunStartResponse)
  
      if run_start_resp.action == OnRunStartAction.DEBUG_RUN:
        # Decorate RunOption to fill in debugger tensor watch specifications.
        decorated_run_options = options or config_pb2.RunOptions()
        run_metadata = run_metadata or config_pb2.RunMetadata()
  
        self._decorate_run_options(decorated_run_options,
                                   run_start_resp.debug_urls)
  
        # Invoke the run() method of the wrapped Session. Catch any TensorFlow
        # runtime errors.
        tf_error = None
        try:
          retvals = self._sess.run(fetches,
                                   feed_dict=feed_dict,
                                   options=decorated_run_options,
                                   run_metadata=run_metadata)
        except errors.OpError as op_error:
          tf_error = op_error
          retvals = op_error
  
        run_end_req = OnRunEndRequest(
            run_start_resp.action,
            run_metadata=run_metadata,
            client_graph_def=self._sess.graph.as_graph_def(),
            tf_error=tf_error)
  
      elif (run_start_resp.action == OnRunStartAction.NON_DEBUG_RUN or
            run_start_resp.action == OnRunStartAction.INVOKE_STEPPER):
        if run_start_resp.action == OnRunStartAction.INVOKE_STEPPER:
          retvals = self.invoke_node_stepper(
              stepper.NodeStepper(self._sess, fetches, feed_dict),
              restore_variable_values_on_exit=True)
  
        # Invoke run() method of the wrapped session.
        retvals = self._sess.run(
            fetches,
            feed_dict=feed_dict,
            options=options,
            run_metadata=run_metadata)
  
        # Prepare arg for the on-run-end callback.
        run_end_req = OnRunEndRequest(run_start_resp.action)
      else:
        raise ValueError(
            "Invalid OnRunStartAction value: %s" % run_start_resp.action)
  
      # Invoke on-run-end callback and obtain response.
      run_end_resp = self.on_run_end(run_end_req)
      _check_type(run_end_resp, OnRunEndResponse)
      # Currently run_end_resp is only a placeholder. No action is taken on it.
  
      return retvals
  
    def partial_run_setup(self, fetches, feeds=None):
      """Sets up the feeds and fetches for partial runs in the session."""
      raise NotImplementedError(
          "partial_run_setup is not implemented for debug-wrapper sessions.")
  
    def partial_run(self, handle, fetches, feed_dict=None):
      raise NotImplementedError(
          "partial_run is not implemented for debug-wrapper sessions.")
  
    def _decorate_run_options(self, run_options, debug_urls):
      """Modify a RunOptions object for debug tensor watching.
  
      Specifies request for outputting partition graphs. Adds
      debug_tensor_watch_opts with proper debug URLs.
  
      Args:
        run_options: (RunOptions) the modified RunOptions object.
        debug_urls: (list of str) debug URLs to be entered in run_options.
          debug_tensor_watch_opts.
      """
  
      run_options.output_partition_graphs = True
      debug_utils.watch_graph(
          run_options, self._sess.graph, debug_urls=debug_urls)
  
    @abc.abstractmethod
    def on_session_init(self, request):
      """Callback invoked during construction of the debug-wrapper session.
  
      This is a blocking callback.
      The invocation happens right before the constructor ends.
  
      Args:
        request: (`OnSessionInitRequest`) callback request carrying information
          such as the session being wrapped.
  
      Returns:
        An instance of `OnSessionInitResponse`.
      """
  
    @abc.abstractmethod
    def on_run_start(self, request):
      """Callback invoked on run() calls to the debug-wrapper session.
  
      This is a blocking callback.
      The invocation happens after the wrapper's run() call is entered,
      after an increment of run call counter.
  
      Args:
        request: (`OnRunStartRequest`) callback request object carrying
          information about the run call such as the fetches, feed dict, run
          options, run metadata, and how many `run()` calls to this wrapper
          session have occurred.
  
      Returns:
        An instance of `OnRunStartResponse`, carrying information to
          1) direct the wrapper session to perform a specified action (e.g., run
            with or without debug tensor watching, invoking the stepper.)
          2) debug URLs used to watch the tensors.
      """
  
    @abc.abstractmethod
    def on_run_end(self, request):
      """Callback invoked on run() calls to the debug-wrapper session.
  
      This is a blocking callback.
      The invocation happens right before the wrapper exits its run() call.
  
      Args:
        request: (`OnRunEndRequest`) callback request object carrying information
          such as the actual action performed by the session wrapper for the
          run() call.
  
      Returns:
        An instance of `OnRunStartResponse`.
      """
  
    def __enter__(self):
      return self._sess.__enter__()
  
    def __exit__(self, exec_type, exec_value, exec_tb):
      self._sess.__exit__(exec_type, exec_value, exec_tb)
  
    def close(self):
      self._sess.close()
  
    # TODO(cais): Add _node_name_regex_whitelist and
    #   _node_op_type_regex_whitelist.
  
    @abc.abstractmethod
    def invoke_node_stepper(self,
                            node_stepper,
                            restore_variable_values_on_exit=True):
      """Callback invoked when the client intends to step through graph nodes.
  
      Args:
        node_stepper: (stepper.NodeStepper) An instance of NodeStepper to be used
          in this stepping session.
        restore_variable_values_on_exit: (bool) Whether any variables whose values
          have been altered during this node-stepper invocation should be restored
          to their old values when this invocation ends.
  
      Returns:
        The same return values as the `Session.run()` call on the same fetches as
          the NodeStepper.
      """
   # Copyright 2016 The TensorFlow Authors. All Rights Reserved.
  #
  # Licensed under the Apache License, Version 2.0 (the "License");
  # you may not use this file except in compliance with the License.
  # You may obtain a copy of the License at
  #
  #     http://www.apache.org/licenses/LICENSE-2.0
  #
  # Unless required by applicable law or agreed to in writing, software
  # distributed under the License is distributed on an "AS IS" BASIS,
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  # See the License for the specific language governing permissions and
  # limitations under the License.
  # ==============================================================================
  """The InverseGamma distribution class."""
  
  from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  
  import numpy as np
  
  from tensorflow.contrib.distributions.python.ops import distribution
  from tensorflow.contrib.distributions.python.ops import distribution_util
  from tensorflow.python.framework import constant_op
  from tensorflow.python.framework import dtypes
  from tensorflow.python.framework import ops
  from tensorflow.python.framework import tensor_shape
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import check_ops
  from tensorflow.python.ops import control_flow_ops
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import nn
  from tensorflow.python.ops import random_ops
  
  
  class InverseGamma(distribution.Distribution):
    """The `InverseGamma` distribution with parameter alpha and beta.
  
    The parameters are the shape and inverse scale parameters alpha, beta.
  
    The PDF of this distribution is:
  
    ```pdf(x) = (beta^alpha)/Gamma(alpha)(x^(-alpha-1))e^(-beta/x), x > 0```
  
    and the CDF of this distribution is:
  
    ```cdf(x) =  GammaInc(alpha, beta / x) / Gamma(alpha), x > 0```
  
    where GammaInc is the upper incomplete Gamma function.
  
    Examples:
  
    ```python
    dist = InverseGamma(alpha=3.0, beta=2.0)
    dist2 = InverseGamma(alpha=[3.0, 4.0], beta=[2.0, 3.0])
    ```
  
    """
  
    def __init__(self,
                 alpha,
                 beta,
                 validate_args=False,
                 allow_nan_stats=True,
                 name="InverseGamma"):
      """Construct InverseGamma distributions with parameters `alpha` and `beta`.
  
      The parameters `alpha` and `beta` must be shaped in a way that supports
      broadcasting (e.g. `alpha + beta` is a valid operation).
  
      Args:
        alpha: Floating point tensor, the shape params of the
          distribution(s).
          alpha must contain only positive values.
        beta: Floating point tensor, the scale params of the distribution(s).
          beta must contain only positive values.
        validate_args: `Boolean`, default `False`.  Whether to assert that
          `a > 0`, `b > 0`, and that `x > 0` in the methods `prob(x)` and
          `log_prob(x)`.  If `validate_args` is `False` and the inputs are
          invalid, correct behavior is not guaranteed.
        allow_nan_stats: `Boolean`, default `True`.  If `False`, raise an
          exception if a statistic (e.g. mean/mode/etc...) is undefined for any
          batch member.  If `True`, batch members with valid parameters leading to
          undefined statistics will return NaN for this statistic.
        name: The name to prepend to all ops created by this distribution.
  
      Raises:
        TypeError: if `alpha` and `beta` are different dtypes.
      """
      parameters = locals()
      parameters.pop("self")
      with ops.name_scope(name, values=[alpha, beta]) as ns:
        with ops.control_dependencies([
            check_ops.assert_positive(alpha),
            check_ops.assert_positive(beta),
        ] if validate_args else []):
          self._alpha = array_ops.identity(alpha, name="alpha")
          self._beta = array_ops.identity(beta, name="beta")
      super(InverseGamma, self).__init__(
          dtype=self._alpha.dtype,
          validate_args=validate_args,
          allow_nan_stats=allow_nan_stats,
          is_continuous=True,
          is_reparameterized=False,
          parameters=parameters,
          graph_parents=[self._alpha, self._beta],
          name=ns)
  
    @staticmethod
    def _param_shapes(sample_shape):
      return dict(
          zip(("alpha", "beta"), ([ops.convert_to_tensor(
              sample_shape, dtype=dtypes.int32)] * 2)))
  
    @property
    def alpha(self):
      """Shape parameter."""
      return self._alpha
  
    @property
    def beta(self):
      """Scale parameter."""
      return self._beta
  
    def _batch_shape(self):
      return array_ops.broadcast_dynamic_shape(
          array_ops.shape(self.alpha), array_ops.shape(self.beta))
  
    def _get_batch_shape(self):
      return array_ops.broadcast_static_shape(
          self.alpha.get_shape(), self.beta.get_shape())
  
    def _event_shape(self):
      return constant_op.constant([], dtype=dtypes.int32)
  
    def _get_event_shape(self):
      return tensor_shape.scalar()
  
    def _sample_n(self, n, seed=None):
      """See the documentation for tf.random_gamma for more details."""
      return 1. / random_ops.random_gamma([n], self.alpha, beta=self.beta,
                                          dtype=self.dtype, seed=seed)
  
    def _log_prob(self, x):
      x = control_flow_ops.with_dependencies([check_ops.assert_positive(x)] if
                                             self.validate_args else [], x)
      return (self.alpha * math_ops.log(self.beta) -
              math_ops.lgamma(self.alpha) -
              (self.alpha + 1.) * math_ops.log(x) - self.beta / x)
  
    def _prob(self, x):
      return math_ops.exp(self._log_prob(x))
  
    def _log_cdf(self, x):
      return math_ops.log(self._cdf(x))
  
    def _cdf(self, x):
      x = control_flow_ops.with_dependencies([check_ops.assert_positive(x)] if
                                             self.validate_args else [], x)
      # Note that igammac returns the upper regularized incomplete gamma
      # function Q(a, x), which is what we want for the CDF.
      return math_ops.igammac(self.alpha, self.beta / x)
  
    @distribution_util.AppendDocstring(
        """This is defined to be
  
        ```
        entropy = alpha - log(beta) + log(Gamma(alpha))
        + (1-alpha)digamma(alpha)
        ```
  
        where digamma(alpha) is the digamma function.""")
    def _entropy(self):
      return (self.alpha +
              math_ops.log(self.beta) +
              math_ops.lgamma(self.alpha) -
              (1. + self.alpha) * math_ops.digamma(self.alpha))
  
    @distribution_util.AppendDocstring(
        """The mean of an inverse gamma distribution is `beta / (alpha - 1)`,
        when `alpha > 1`, and `NaN` otherwise.  If `self.allow_nan_stats` is
        `False`, an exception will be raised rather than returning `NaN`""")
    def _mean(self):
      mean = self.beta / (self.alpha - 1.)
      if self.allow_nan_stats:
        nan = np.array(np.nan, dtype=self.dtype.as_numpy_dtype())
        return array_ops.where(
            self.alpha > 1., mean,
            array_ops.fill(self.batch_shape(), nan, name="nan"))
      else:
        return control_flow_ops.with_dependencies([
            check_ops.assert_less(
                array_ops.ones((), self.dtype), self.alpha,
                message="mean not defined for components of self.alpha <= 1"),
        ], mean)
  
    @distribution_util.AppendDocstring(
        """Variance for inverse gamma is defined only for `alpha > 2`. If
        `self.allow_nan_stats` is `False`, an exception will be raised rather
        than returning `NaN`.""")
    def _variance(self):
      var = (math_ops.square(self.beta) /
             (math_ops.square(self.alpha - 1.) * (self.alpha - 2.)))
      if self.allow_nan_stats:
        nan = np.array(np.nan, dtype=self.dtype.as_numpy_dtype())
        return array_ops.where(
            self.alpha > 2., var,
            array_ops.fill(self.batch_shape(), nan, name="nan"))
      else:
        return control_flow_ops.with_dependencies([
            check_ops.assert_less(
                constant_op.constant(2., dtype=self.dtype), self.alpha,
                message="variance not defined for components of alpha <= 2"),
        ], var)
  
    def _mode(self):
      """The mode of an inverse gamma distribution is `beta / (alpha + 1)`."""
      return self.beta / (self.alpha + 1.)
  
  
  class InverseGammaWithSoftplusAlphaBeta(InverseGamma):
    """Inverse Gamma with softplus applied to `alpha` and `beta`."""
  
    def __init__(self,
                 alpha,
                 beta,
                 validate_args=False,
                 allow_nan_stats=True,
                 name="InverseGammaWithSoftplusAlphaBeta"):
      parameters = locals()
      parameters.pop("self")
      with ops.name_scope(name, values=[alpha, beta]) as ns:
        super(InverseGammaWithSoftplusAlphaBeta, self).__init__(
            alpha=nn.softplus(alpha, name="softplus_alpha"),
            beta=nn.softplus(beta, name="softplus_gamma"),
            validate_args=validate_args,
            allow_nan_stats=allow_nan_stats,
            name=ns)
      self._parameters = parameters
   # Copyright 2015 The TensorFlow Authors. All Rights Reserved.
  #
  # Licensed under the Apache License, Version 2.0 (the "License");
  # you may not use this file except in compliance with the License.
  # You may obtain a copy of the License at
  #
  #     http://www.apache.org/licenses/LICENSE-2.0
  #
  # Unless required by applicable law or agreed to in writing, software
  # distributed under the License is distributed on an "AS IS" BASIS,
  # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  # See the License for the specific language governing permissions and
  # limitations under the License.
  # ==============================================================================
  """Tests for RNN cells."""
  
  from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  
  import functools
  import sys
  
  # TODO: #6568 Remove this hack that makes dlopen() not crash.
  if hasattr(sys, "getdlopenflags") and hasattr(sys, "setdlopenflags"):
    import ctypes
    sys.setdlopenflags(sys.getdlopenflags() | ctypes.RTLD_GLOBAL)
  
  import numpy as np
  
  # TODO(ebrevdo): Remove once _linear is fully deprecated.
  # pylint: disable=protected-access
  
  from tensorflow.contrib.rnn.python.ops import core_rnn_cell_impl
  from tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl import _linear as linear
  from tensorflow.python.framework import dtypes
  from tensorflow.python.framework import ops
  from tensorflow.python.ops import array_ops
  from tensorflow.python.ops import init_ops
  from tensorflow.python.ops import math_ops
  from tensorflow.python.ops import random_ops
  from tensorflow.python.ops import rnn
  from tensorflow.python.ops import variable_scope
  from tensorflow.python.ops import variables as variables_lib
  from tensorflow.python.platform import test
  
  # pylint: enable=protected-access
  
  
  class RNNCellTest(test.TestCase):
  
    def testLinear(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(1.0)):
          x = array_ops.zeros([1, 2])
          l = linear([x], 2, False)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([l], {x.name: np.array([[1., 2.]])})
          self.assertAllClose(res[0], [[3.0, 3.0]])
  
          # Checks prevent you from accidentally creating a shared function.
          with self.assertRaises(ValueError):
            l1 = linear([x], 2, False)
  
          # But you can create a new one in a new scope and share the variables.
          with variable_scope.variable_scope("l1") as new_scope:
            l1 = linear([x], 2, False)
          with variable_scope.variable_scope(new_scope, reuse=True):
            linear([l1], 2, False)
          self.assertEqual(len(variables_lib.trainable_variables()), 2)
  
    def testBasicRNNCell(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 2])
          g, _ = core_rnn_cell_impl.BasicRNNCell(2)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g], {x.name: np.array([[1., 1.]]),
                    m.name: np.array([[0.1, 0.1]])})
          self.assertEqual(res[0].shape, (1, 2))
  
    def testGRUCell(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 2])
          g, _ = core_rnn_cell_impl.GRUCell(2)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g], {x.name: np.array([[1., 1.]]),
                    m.name: np.array([[0.1, 0.1]])})
          # Smoke test
          self.assertAllClose(res[0], [[0.175991, 0.175991]])
        with variable_scope.variable_scope(
            "other", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros(
              [1, 3])  # Test GRUCell with input_size != num_units.
          m = array_ops.zeros([1, 2])
          g, _ = core_rnn_cell_impl.GRUCell(2)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g],
              {x.name: np.array([[1., 1., 1.]]),
               m.name: np.array([[0.1, 0.1]])})
          # Smoke test
          self.assertAllClose(res[0], [[0.156736, 0.156736]])
  
    def testBasicLSTMCell(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 8])
          g, out_m = core_rnn_cell_impl.MultiRNNCell(
              [core_rnn_cell_impl.BasicLSTMCell(
                  2, state_is_tuple=False)] * 2,
              state_is_tuple=False)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g, out_m],
              {x.name: np.array([[1., 1.]]),
               m.name: 0.1 * np.ones([1, 8])})
          self.assertEqual(len(res), 2)
          variables = variables_lib.global_variables()
          self.assertEqual(4, len(variables))
          self.assertEquals(variables[0].op.name,
                            "root/multi_rnn_cell/cell_0/basic_lstm_cell/weights")
          self.assertEquals(variables[1].op.name,
                            "root/multi_rnn_cell/cell_0/basic_lstm_cell/biases")
          self.assertEquals(variables[2].op.name,
                            "root/multi_rnn_cell/cell_1/basic_lstm_cell/weights")
          self.assertEquals(variables[3].op.name,
                            "root/multi_rnn_cell/cell_1/basic_lstm_cell/biases")
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res[0], [[0.24024698, 0.24024698]])
          expected_mem = np.array([[
              0.68967271, 0.68967271, 0.44848421, 0.44848421, 0.39897051,
              0.39897051, 0.24024698, 0.24024698
          ]])
          self.assertAllClose(res[1], expected_mem)
        with variable_scope.variable_scope(
            "other", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros(
              [1, 3])  # Test BasicLSTMCell with input_size != num_units.
          m = array_ops.zeros([1, 4])
          g, out_m = core_rnn_cell_impl.BasicLSTMCell(
              2, state_is_tuple=False)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g, out_m],
              {x.name: np.array([[1., 1., 1.]]),
               m.name: 0.1 * np.ones([1, 4])})
          self.assertEqual(len(res), 2)
  
    def testBasicLSTMCellStateTupleType(self):
      with self.test_session():
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m0 = (array_ops.zeros([1, 2]),) * 2
          m1 = (array_ops.zeros([1, 2]),) * 2
          cell = core_rnn_cell_impl.MultiRNNCell(
              [core_rnn_cell_impl.BasicLSTMCell(2)] * 2, state_is_tuple=True)
          self.assertTrue(isinstance(cell.state_size, tuple))
          self.assertTrue(
              isinstance(cell.state_size[0], core_rnn_cell_impl.LSTMStateTuple))
          self.assertTrue(
              isinstance(cell.state_size[1], core_rnn_cell_impl.LSTMStateTuple))
  
          # Pass in regular tuples
          _, (out_m0, out_m1) = cell(x, (m0, m1))
          self.assertTrue(isinstance(out_m0, core_rnn_cell_impl.LSTMStateTuple))
          self.assertTrue(isinstance(out_m1, core_rnn_cell_impl.LSTMStateTuple))
  
          # Pass in LSTMStateTuples
          variable_scope.get_variable_scope().reuse_variables()
          zero_state = cell.zero_state(1, dtypes.float32)
          self.assertTrue(isinstance(zero_state, tuple))
          self.assertTrue(
              isinstance(zero_state[0], core_rnn_cell_impl.LSTMStateTuple))
          self.assertTrue(
              isinstance(zero_state[1], core_rnn_cell_impl.LSTMStateTuple))
          _, (out_m0, out_m1) = cell(x, zero_state)
          self.assertTrue(isinstance(out_m0, core_rnn_cell_impl.LSTMStateTuple))
          self.assertTrue(isinstance(out_m1, core_rnn_cell_impl.LSTMStateTuple))
  
    def testBasicLSTMCellWithStateTuple(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m0 = array_ops.zeros([1, 4])
          m1 = array_ops.zeros([1, 4])
          cell = core_rnn_cell_impl.MultiRNNCell(
              [core_rnn_cell_impl.BasicLSTMCell(
                  2, state_is_tuple=False)] * 2,
              state_is_tuple=True)
          g, (out_m0, out_m1) = cell(x, (m0, m1))
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([g, out_m0, out_m1], {
              x.name: np.array([[1., 1.]]),
              m0.name: 0.1 * np.ones([1, 4]),
              m1.name: 0.1 * np.ones([1, 4])
          })
          self.assertEqual(len(res), 3)
          # The numbers in results were not calculated, this is just a smoke test.
          # Note, however, these values should match the original
          # version having state_is_tuple=False.
          self.assertAllClose(res[0], [[0.24024698, 0.24024698]])
          expected_mem0 = np.array(
              [[0.68967271, 0.68967271, 0.44848421, 0.44848421]])
          expected_mem1 = np.array(
              [[0.39897051, 0.39897051, 0.24024698, 0.24024698]])
          self.assertAllClose(res[1], expected_mem0)
          self.assertAllClose(res[2], expected_mem1)
  
    def testLSTMCell(self):
      with self.test_session() as sess:
        num_units = 8
        num_proj = 6
        state_size = num_units + num_proj
        batch_size = 3
        input_size = 2
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([batch_size, input_size])
          m = array_ops.zeros([batch_size, state_size])
          cell = core_rnn_cell_impl.LSTMCell(
              num_units=num_units,
              num_proj=num_proj,
              forget_bias=1.0,
              state_is_tuple=False)
          output, state = cell(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([output, state], {
              x.name: np.array([[1., 1.], [2., 2.], [3., 3.]]),
              m.name: 0.1 * np.ones((batch_size, state_size))
          })
          self.assertEqual(len(res), 2)
          # The numbers in results were not calculated, this is mostly just a
          # smoke test.
          self.assertEqual(res[0].shape, (batch_size, num_proj))
          self.assertEqual(res[1].shape, (batch_size, state_size))
          # Different inputs so different outputs and states
          for i in range(1, batch_size):
            self.assertTrue(
                float(np.linalg.norm((res[0][0, :] - res[0][i, :]))) > 1e-6)
            self.assertTrue(
                float(np.linalg.norm((res[1][0, :] - res[1][i, :]))) > 1e-6)
  
    def testLSTMCellVariables(self):
      with self.test_session():
        num_units = 8
        num_proj = 6
        state_size = num_units + num_proj
        batch_size = 3
        input_size = 2
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([batch_size, input_size])
          m = array_ops.zeros([batch_size, state_size])
          cell = core_rnn_cell_impl.LSTMCell(
              num_units=num_units,
              num_proj=num_proj,
              forget_bias=1.0,
              state_is_tuple=False)
          cell(x, m)  # Execute to create variables
        variables = variables_lib.global_variables()
        self.assertEquals(variables[0].op.name, "root/lstm_cell/weights")
        self.assertEquals(variables[1].op.name, "root/lstm_cell/biases")
        self.assertEquals(variables[2].op.name,
                          "root/lstm_cell/projection/weights")
  
    def testOutputProjectionWrapper(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 3])
          m = array_ops.zeros([1, 3])
          cell = core_rnn_cell_impl.OutputProjectionWrapper(
              core_rnn_cell_impl.GRUCell(3), 2)
          g, new_m = cell(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([g, new_m], {
              x.name: np.array([[1., 1., 1.]]),
              m.name: np.array([[0.1, 0.1, 0.1]])
          })
          self.assertEqual(res[1].shape, (1, 3))
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res[0], [[0.231907, 0.231907]])
  
    def testInputProjectionWrapper(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 3])
          cell = core_rnn_cell_impl.InputProjectionWrapper(
              core_rnn_cell_impl.GRUCell(3), num_proj=3)
          g, new_m = cell(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g, new_m],
              {x.name: np.array([[1., 1.]]),
               m.name: np.array([[0.1, 0.1, 0.1]])})
          self.assertEqual(res[1].shape, (1, 3))
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res[0], [[0.154605, 0.154605, 0.154605]])
  
    def testDropoutWrapper(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 3])
          m = array_ops.zeros([1, 3])
          keep = array_ops.zeros([]) + 1
          g, new_m = core_rnn_cell_impl.DropoutWrapper(
              core_rnn_cell_impl.GRUCell(3), keep, keep)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([g, new_m], {
              x.name: np.array([[1., 1., 1.]]),
              m.name: np.array([[0.1, 0.1, 0.1]])
          })
          self.assertEqual(res[1].shape, (1, 3))
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res[0], [[0.154605, 0.154605, 0.154605]])
  
    def testEmbeddingWrapper(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 1], dtype=dtypes.int32)
          m = array_ops.zeros([1, 2])
          embedding_cell = core_rnn_cell_impl.EmbeddingWrapper(
              core_rnn_cell_impl.GRUCell(2),
              embedding_classes=3,
              embedding_size=2)
          self.assertEqual(embedding_cell.output_size, 2)
          g, new_m = embedding_cell(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g, new_m],
              {x.name: np.array([[1]]),
               m.name: np.array([[0.1, 0.1]])})
          self.assertEqual(res[1].shape, (1, 2))
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res[0], [[0.17139, 0.17139]])
  
    def testEmbeddingWrapperWithDynamicRnn(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope("root"):
          inputs = ops.convert_to_tensor([[[0], [0]]], dtype=dtypes.int64)
          input_lengths = ops.convert_to_tensor([2], dtype=dtypes.int64)
          embedding_cell = core_rnn_cell_impl.EmbeddingWrapper(
              core_rnn_cell_impl.BasicLSTMCell(
                  1, state_is_tuple=True),
              embedding_classes=1,
              embedding_size=2)
          outputs, _ = rnn.dynamic_rnn(
              cell=embedding_cell,
              inputs=inputs,
              sequence_length=input_lengths,
              dtype=dtypes.float32)
          sess.run([variables_lib.global_variables_initializer()])
          # This will fail if output's dtype is inferred from input's.
          sess.run(outputs)
  
    def testMultiRNNCell(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 4])
          _, ml = core_rnn_cell_impl.MultiRNNCell(
              [core_rnn_cell_impl.GRUCell(2)] * 2, state_is_tuple=False)(x, m)
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(ml, {
              x.name: np.array([[1., 1.]]),
              m.name: np.array([[0.1, 0.1, 0.1, 0.1]])
          })
          # The numbers in results were not calculated, this is just a smoke test.
          self.assertAllClose(res, [[0.175991, 0.175991, 0.13248, 0.13248]])
  
    def testMultiRNNCellWithStateTuple(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m_bad = array_ops.zeros([1, 4])
          m_good = (array_ops.zeros([1, 2]), array_ops.zeros([1, 2]))
  
          # Test incorrectness of state
          with self.assertRaisesRegexp(ValueError, "Expected state .* a tuple"):
            core_rnn_cell_impl.MultiRNNCell(
                [core_rnn_cell_impl.GRUCell(2)] * 2,
                state_is_tuple=True)(x, m_bad)
  
          _, ml = core_rnn_cell_impl.MultiRNNCell(
              [core_rnn_cell_impl.GRUCell(2)] * 2, state_is_tuple=True)(x, m_good)
  
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(ml, {
              x.name: np.array([[1., 1.]]),
              m_good[0].name: np.array([[0.1, 0.1]]),
              m_good[1].name: np.array([[0.1, 0.1]])
          })
  
          # The numbers in results were not calculated, this is just a
          # smoke test.  However, these numbers should match those of
          # the test testMultiRNNCell.
          self.assertAllClose(res[0], [[0.175991, 0.175991]])
          self.assertAllClose(res[1], [[0.13248, 0.13248]])
  
  
  class SlimRNNCellTest(test.TestCase):
  
    def testBasicRNNCell(self):
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          x = array_ops.zeros([1, 2])
          m = array_ops.zeros([1, 2])
          my_cell = functools.partial(basic_rnn_cell, num_units=2)
          # pylint: disable=protected-access
          g, _ = core_rnn_cell_impl._SlimRNNCell(my_cell)(x, m)
          # pylint: enable=protected-access
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run(
              [g], {x.name: np.array([[1., 1.]]),
                    m.name: np.array([[0.1, 0.1]])})
          self.assertEqual(res[0].shape, (1, 2))
  
    def testBasicRNNCellMatch(self):
      batch_size = 32
      input_size = 100
      num_units = 10
      with self.test_session() as sess:
        with variable_scope.variable_scope(
            "root", initializer=init_ops.constant_initializer(0.5)):
          inputs = random_ops.random_uniform((batch_size, input_size))
          _, initial_state = basic_rnn_cell(inputs, None, num_units)
          my_cell = functools.partial(basic_rnn_cell, num_units=num_units)
          # pylint: disable=protected-access
          slim_cell = core_rnn_cell_impl._SlimRNNCell(my_cell)
          # pylint: enable=protected-access
          slim_outputs, slim_state = slim_cell(inputs, initial_state)
          rnn_cell = core_rnn_cell_impl.BasicRNNCell(num_units)
          variable_scope.get_variable_scope().reuse_variables()
          outputs, state = rnn_cell(inputs, initial_state)
          self.assertEqual(slim_outputs.get_shape(), outputs.get_shape())
          self.assertEqual(slim_state.get_shape(), state.get_shape())
          sess.run([variables_lib.global_variables_initializer()])
          res = sess.run([slim_outputs, slim_state, outputs, state])
          self.assertAllClose(res[0], res[2])
          self.assertAllClose(res[1], res[3])
  
  
  def basic_rnn_cell(inputs, state, num_units, scope=None):
    if state is None:
      if inputs is not None:
        batch_size = inputs.get_shape()[0]
        dtype = inputs.dtype
      else:
        batch_size = 0
        dtype = dtypes.float32
      init_output = array_ops.zeros(
          array_ops.stack([batch_size, num_units]), dtype=dtype)
      init_state = array_ops.zeros(
          array_ops.stack([batch_size, num_units]), dtype=dtype)
      init_output.set_shape([batch_size, num_units])
      init_state.set_shape([batch_size, num_units])
      return init_output, init_state
    else:
      with variable_scope.variable_scope(scope, "basic_rnn_cell",
                                         [inputs, state]):
        output = math_ops.tanh(linear([inputs, state], num_units, True))
      return output, output
  
  
  if __name__ == "__main__":
    test.main()
  
apis: tensorflow.core.protobuf.config_pb2.RunOptions tensorflow.python.debug.debug_utils.watch_graph tensorflow.python.debug.stepper.NodeStepper tensorflow.core.protobuf.config_pb2.RunMetadata
apis: tensorflow.python.ops.math_ops.square tensorflow.python.ops.nn.softplus tensorflow.python.ops.array_ops.identity tensorflow.python.framework.tensor_shape.scalar tensorflow.contrib.distributions.python.ops.distribution_util.AppendDocstring tensorflow.python.ops.array_ops.shape tensorflow.python.ops.math_ops.log tensorflow.python.ops.array_ops.ones tensorflow.python.framework.ops.name_scope tensorflow.python.ops.math_ops.lgamma tensorflow.python.ops.math_ops.igammac tensorflow.python.ops.check_ops.assert_positive tensorflow.python.ops.random_ops.random_gamma tensorflow.python.framework.ops.convert_to_tensor tensorflow.python.ops.math_ops.digamma tensorflow.python.framework.constant_op.constant
apis: numpy.ones tensorflow.python.ops.variable_scope.variable_scope tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.LSTMCell tensorflow.python.ops.init_ops.constant_initializer tensorflow.python.ops.rnn.dynamic_rnn tensorflow.python.ops.random_ops.random_uniform tensorflow.python.ops.array_ops.zeros tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicRNNCell tensorflow.python.ops.variables.global_variables_initializer tensorflow.python.ops.variables.trainable_variables tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl._SlimRNNCell tensorflow.python.framework.ops.convert_to_tensor numpy.linalg.norm tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl._linear tensorflow.python.ops.variable_scope.get_variable_scope tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicLSTMCell tensorflow.python.platform.test.main tensorflow.python.ops.array_ops.stack tensorflow.python.ops.variables.global_variables numpy.array