class_docstr: Function node of the computational graph.
  
  FunctionNode is a class representing a node in a computational graph. The
  node corresponds to an application of a differentiable function to input
  variables.
  
  When a differentiable function is applied to :class:`~chainer.Variable`
  objects,
  it creates an instance of FunctionNode implementation and calls its
  :meth:`apply` method. The :meth:`apply` method basically does the following
  three things.
  
  1. Adding an edge from the function node to the variable node corresponding
     to each input. The node of each input is extracted by
     :attr:`Variable.node <chainer.Variable.node>`.
  2. Computing the output arrays of the function.
  3. Creating a :class:`~chainer.Variable` object for each output array and
     adding an edge from the node of the variable to the function node.
  
  The output variables are then returned.
  
  .. admonition:: Example
  
     Let ``x`` be an instance of :class:`~chainer.Variable` and ``f`` be an
     instance of :class:`FunctionNode` taking only one argument.
     Then the following code
  
     >>> import numpy, chainer
     >>> x = chainer.Variable(numpy.zeros(10))
     >>> f = chainer.functions.math.identity.Identity()
     >>> y = f.apply((x,))[0]
  
     computes a new variable ``y`` and creates backward references. The
     backward references are actually set as per the following diagram::
  
         x.node <--- f <--- y.node
  
     If an application of another function ``g`` occurs as
  
     >>> g = chainer.functions.math.identity.Identity()
     >>> z = g.apply((x,))[0]
  
     then the graph grows with a branch::
  
                  |--- f <--- y.node
         x.node <-+
                  |--- g <--- z.node
  
     Note that the branching is correctly managed on backward computation,
     i.e. the gradients from ``f`` and ``g`` are accumulated to the gradient
     of ``x``.
  
  Every function-node implementation should provide :meth:`forward` and
  :meth:`backward`. Instead of overriding :meth:`forward`, one can also
  implement :meth:`forward_cpu` and :meth:`forward_gpu` when the
  implementations for CPU and GPU arrays are totally different.
  
  Note that the input and output variables are inaccessible from
  :meth:`backward` by default. If it needs accesses to these variables, the
  :meth:`forward` method (or its CPU/GPU variants) has to call
  :meth:`retain_inputs` and :meth:`retain_outputs` appropriately. The
  retained input/output variables can be accessed from :meth:`backward` by
  calling :meth:`get_retained_inputs` and :meth:`get_retained_outputs`.
  
  .. note::
  
     There are two types of differentiable functions in Chainer (since v3).
     The first type is of a function using a subclass of
     :class:`~chainer.Function`,
     which is called *old-style differentiable function*. The second type is
     of a function using a subclass of :class:`FunctionNode`, which is called
     **new-style differentiable function**. There are several advantages on
     using the new-style differentiable function.
  
     - The new-style differentiable function supports *differentiable
       backpropagation*. The backpropagated gradients computed through the
       new-style differentiable functions themselves support further
       backpropagations so that the automatic higher-order differentiation is
       available.
     - The backpropagation of the new-style differentiable function can be
       more computationally efficient because the interface allows an
       implementation to omit the computation of unneeded input gradients.
  
     Note that the new-style differentiable function is the standard way of
     defining a function node of the computational graph in Chainer; old-
     style differentiable functions are implemented as wrappers of the new-
     style differentiable functions.
  
  Attributes:
      ~FunctionNode.inputs: A tuple of the input
          :class:`~chainer.variable.VariableNode` objects.
      ~FunctionNode.outputs: A tuple of weak references to the output
          :class:`~chainer.variable.VariableNode` objects.
      ~FunctionNode.rank (int): An ordinal following the topological order
          of the computational graph.
      ~FunctionNode.stack: Stack trace retrieved at the forward computation.
          The stack trace is available only in the debug mode.
  
  .. versionadded:: 3.0.0
class_docstr_tok: ["Function","node","of","the","computational","graph",".","FunctionNode","is","a","class","representing","a","node","in","a","computational","graph",".","The","node","corresponds","to","an","application","of","a","differentiable","function","to","input","variables",".","When","a","differentiable","function","is","applied","to",":","class",":","`","~chainer.Variable","`","objects",",","it","creates","an","instance","of","FunctionNode","implementation","and","calls","its",":","meth",":","`","apply","`","method",".","The",":","meth",":","`","apply","`","method","basically","does","the","following","three","things",".","1",".","Adding","an","edge","from","the","function","node","to","the","variable","node","corresponding","to","each","input",".","The","node","of","each","input","is","extracted","by",":","attr",":","`","Variable.node","<","chainer.Variable.node",">","`",".","2",".","Computing","the","output","arrays","of","the","function",".","3",".","Creating","a",":","class",":","`","~chainer.Variable","`","object","for","each","output","array","and","adding","an","edge","from","the","node","of","the","variable","to","the","function","node",".","The","output","variables","are","then","returned",".","..","admonition",":",":","Example","Let","``","x","``","be","an","instance","of",":","class",":","`","~chainer.Variable","`","and","``","f","``","be","an","instance","of",":","class",":","`","FunctionNode","`","taking","only","one","argument",".","Then","the","following","code",">",">",">","import","numpy",",","chainer",">",">",">","x","=","chainer.Variable","(","numpy.zeros","(","10",")",")",">",">",">","f","=","chainer.functions.math.identity.Identity","(",")",">",">",">","y","=","f.apply","(","(","x",",",")",")","[","0","]","computes","a","new","variable","``","y","``","and","creates","backward","references",".","The","backward","references","are","actually","set","as","per","the","following","diagram",":",":","x.node","<","--","-","f","<","--","-","y.node","If","an","application","of","another","function","``","g","``","occurs","as",">",">",">","g","=","chainer.functions.math.identity.Identity","(",")",">",">",">","z","=","g.apply","(","(","x",",",")",")","[","0","]","then","the","graph","grows","with","a","branch",":",":","|","--","-","f","<","--","-","y.node","x.node","<","-+","|","--","-","g","<","--","-","z.node","Note","that","the","branching","is","correctly","managed","on","backward","computation",",","i.e",".","the","gradients","from","``","f","``","and","``","g","``","are","accumulated","to","the","gradient","of","``","x","``",".","Every","function-node","implementation","should","provide",":","meth",":","`","forward","`","and",":","meth",":","`","backward","`",".","Instead","of","overriding",":","meth",":","`","forward","`",",","one","can","also","implement",":","meth",":","`","forward_cpu","`","and",":","meth",":","`","forward_gpu","`","when","the","implementations","for","CPU","and","GPU","arrays","are","totally","different",".","Note","that","the","input","and","output","variables","are","inaccessible","from",":","meth",":","`","backward","`","by","default",".","If","it","needs","accesses","to","these","variables",",","the",":","meth",":","`","forward","`","method","(","or","its","CPU\/GPU","variants",")","has","to","call",":","meth",":","`","retain_inputs","`","and",":","meth",":","`","retain_outputs","`","appropriately",".","The","retained","input\/output","variables","can","be","accessed","from",":","meth",":","`","backward","`","by","calling",":","meth",":","`","get_retained_inputs","`","and",":","meth",":","`","get_retained_outputs","`",".","..","note",":",":","There","are","two","types","of","differentiable","functions","in","Chainer","(","since","v3",")",".","The","first","type","is","of","a","function","using","a","subclass","of",":","class",":","`","~chainer.Function","`",",","which","is","called","*","old-style","differentiable","function","*",".","The","second","type","is","of","a","function","using","a","subclass","of",":","class",":","`","FunctionNode","`",",","which","is","called","*","*","new-style","differentiable","function","*","*",".","There","are","several","advantages","on","using","the","new-style","differentiable","function",".","-","The","new-style","differentiable","function","supports","*","differentiable","backpropagation","*",".","The","backpropagated","gradients","computed","through","the","new-style","differentiable","functions","themselves","support","further","backpropagations","so","that","the","automatic","higher-order","differentiation","is","available",".","-","The","backpropagation","of","the","new-style","differentiable","function","can","be","more","computationally","efficient","because","the","interface","allows","an","implementation","to","omit","the","computation","of","unneeded","input","gradients",".","Note","that","the","new-style","differentiable","function","is","the","standard","way","of","defining","a","function","node","of","the","computational","graph","in","Chainer",";","old-","style","differentiable","functions","are","implemented","as","wrappers","of","the","new-","style","differentiable","functions",".","Attributes",":","~FunctionNode.inputs",":","A","tuple","of","the","input",":","class",":","`","~chainer.variable.VariableNode","`","objects",".","~FunctionNode.outputs",":","A","tuple","of","weak","references","to","the","output",":","class",":","`","~chainer.variable.VariableNode","`","objects",".","~FunctionNode.rank","(","int",")",":","An","ordinal","following","the","topological","order","of","the","computational","graph",".","~FunctionNode.stack",":","Stack","trace","retrieved","at","the","forward","computation",".","The","stack","trace","is","available","only","in","the","debug","mode",".","..","versionadded",":",":","3.0.0"]
docstr: Computes output variables and grows the computational graph.
  
  Basic behavior is expressed in the documentation of
  :class:`FunctionNode`.
  
  .. note::
  
     If the :data:`~Variable.data` attributes of the input variables
     exist on a GPU device, that device is made current before calling
     :meth:`forward`, so implementers do not need to take care of device
     selection in most cases.
  
  Args:
      inputs: Tuple of input variables. Each element can be either
          :class:`~chainer.Variable` or :ref:`ndarray`. If the element
          is an ndarray, it is automatically wrapped with
          :class:`~chainer.Variable`.
  
  Returns:
      A tuple of output :class:`~chainer.Variable` objects.
docstr_tok: ["Computes","output","variables","and","grows","the","computational","graph",".","Basic","behavior","is","expressed","in","the","documentation","of",":","class",":","`","FunctionNode","`",".","..","note",":",":","If","the",":","data",":","`","~Variable.data","`","attributes","of","the","input","variables","exist","on","a","GPU","device",",","that","device","is","made","current","before","calling",":","meth",":","`","forward","`",",","so","implementers","do","not","need","to","take","care","of","device","selection","in","most","cases",".","Args",":","inputs",":","Tuple","of","input","variables",".","Each","element","can","be","either",":","class",":","`","~chainer.Variable","`","or",":","ref",":","`","ndarray","`",".","If","the","element","is","an","ndarray",",","it","is","automatically","wrapped","with",":","class",":","`","~chainer.Variable","`",".","Returns",":","A","tuple","of","output",":","class",":","`","~chainer.Variable","`","objects","."]
code: def apply(self, inputs):
      """Computes output variables and grows the computational graph.
  
      Basic behavior is expressed in the documentation of
      :class:`FunctionNode`.
  
      .. note::
  
         If the :data:`~Variable.data` attributes of the input variables
         exist on a GPU device, that device is made current before calling
         :meth:`forward`, so implementers do not need to take care of device
         selection in most cases.
  
      Args:
          inputs: Tuple of input variables. Each element can be either
              :class:`~chainer.Variable` or :ref:`ndarray`. If the element
              is an ndarray, it is automatically wrapped with
              :class:`~chainer.Variable`.
  
      Returns:
          A tuple of output :class:`~chainer.Variable` objects.
  
      """
      chainerx_in_data = None
      chainerx_device = None
      is_chainerx, in_data = _extract_apply_in_data(inputs)
  
      utils._check_arrays_forward_compatible(in_data, self.label)
  
      if is_chainerx:
          # Try ChainerX C++ implementation.
          # If it's supported, the output arrays are wrapped with Variables
          # and returned.
          # If not supported, FunctionNode.forward_chainerx should return
          # Fallback.
          # In that case the input arrays are converted to numpy.ndarray
          # or cupy.ndarray (depending on the ChainerX backend) and
          # forward computation falls back to the conventional
          # FunctionNode.forward() implementaion.
          outputs = self.forward_chainerx(in_data)
  
          if outputs is not chainer.Fallback:
              # Supported. Wrap with variables and return
              assert isinstance(outputs, tuple)
              return tuple(
                  [
                      variable.Variable._init_unchecked(
                          y,
                          requires_grad=y.is_backprop_required(),
                          is_chainerx_array=True,
                      )
                      for y in outputs
                  ]
              )
  
          # Fall back to FunctionNode.forward()
          (
              chainerx_in_data,
              in_data,
              chainerx_device,
          ) = self._chainerx_apply_fallback_preprocess(in_data, inputs)
          self._is_chainerx_fallback_mode = True
          self.chainerx_device = chainerx_device
  
      is_debug = chainer.is_debug()
      if is_debug:
          # Keep stack trace for debug
          self.stack = traceback.extract_stack()
  
      input_vars = [chainer.as_variable(x) for x in inputs]
      self._input_layouts = tuple([x.layout for x in input_vars])
  
      if configuration.config.type_check:
          self._check_data_type_forward(in_data)
  
      self.check_layout_forward(input_vars)
  
      # Call preprocess hooks
      hooks = chainer.get_function_hooks()
      if self._n_local_function_hooks > 0:
          hooks = collections.OrderedDict(hooks)
          hooks.update(self.local_function_hooks)
      hooks = hooks.values()  # avoid six for performance
  
      for hook in hooks:
          hook.forward_preprocess(self, in_data)
  
      # Forward propagation
      with chainer.using_device(
          backend.get_device_from_array(*in_data)
      ):
          self._input_indexes_to_retain = None
          self._output_indexes_to_retain = None
          if chainer.config.schedule_func is not None:
              outputs = static_forward_optimizations(self, in_data)
          elif self._is_chainerx_fallback_mode:
              # In ChainerX fallback, __class__ is temporarily replaced with
              # the fabricated one with automatic attirbute fallback.
              with _chainerx_attribute_fallback(self, chainerx_device):
                  outputs = self.forward(in_data)
          else:
              # In normal case, simply run the forward method.
              outputs = self.forward(in_data)
  
      # Check for output array types
      if not isinstance(outputs, tuple):
          raise TypeError(
              "forward output must be a tuple ({})\n"
              "Actual: {}".format(self.label, type(outputs))
          )
  
      if self.is_elementwise:
          if not all([y.shape == outputs[0].shape for y in outputs]):
              raise RuntimeError(
                  "An elementwise function returned outputs with "
                  "different shapes.\n"
                  "Function: {}\n"
                  "Input shapes: {}\n"
                  "Output shapes: {}".format(
                      self.label,
                      ", ".join(repr(x.shape) for x in outputs),
                      ", ".join(repr(y.shape) for y in outputs),
                  )
              )
  
      if not chainer.is_arrays_compatible(outputs):
          if not all(
              isinstance(y, chainer.get_array_types()) for y in outputs
          ):
              raise TypeError(
                  "forward output must be a tuple of ndarrays.\n"
                  "Function: {}\n"
                  "Actual output types: {}".format(
                      self.label, tuple(type(y) for y in outputs)
                  )
              )
  
          raise TypeError(
              "incompatible array types are mixed in the forward output "
              "({}).\n"
              "Actual: {}".format(
                  self.label, ", ".join(str(type(x)) for x in outputs)
              )
          )
  
      # If output layouts is not specified, assign the default layouts.
      if self.is_elementwise:
          assert self._output_layouts is None
          layout = self._input_layouts[0]
          self._output_layouts = (layout,) * len(outputs)
      elif self._output_layouts is None:
          self._output_layouts = (None,) * len(outputs)
  
      # Call postprocess hooks
      for hook in hooks:
          hook.forward_postprocess(self, in_data)
  
      # NaN check of output values
      if is_debug:
          for out in outputs:
              if out is not None and chainer.backend._contains_nan(out):
                  msg = (
                      "NaN is detected on forward computation of "
                      "{}".format(self.label)
                  )
                  raise RuntimeError(msg)
  
      self._output_count = len(outputs)
  
      if self._is_chainerx_fallback_mode:
          ret = self._chainerx_apply_fallback_postprocess(
              chainerx_device, chainerx_in_data, inputs, outputs
          )
  
      else:
          requires_grad = any([x.requires_grad for x in input_vars])
  
          ret = tuple(
              [
                  variable.Variable(
                      y, requires_grad=requires_grad, layout=layout
                  )
                  for y, layout in zip(outputs, self.output_layouts)
              ]
          )
  
          if configuration.config.enable_backprop:
              # Topological ordering
              self.rank = (
                  max([x.rank for x in input_vars]) if input_vars else 0
              )
              # Add backward edges
              for y in ret:
                  y.creator_node = self
              self.inputs = tuple([x.node for x in input_vars])
              # Add forward edges (must be weak references)
              self.outputs = tuple([weakref.ref(y.node) for y in ret])
  
              if self._input_indexes_to_retain is not None:
                  for index in self._input_indexes_to_retain:
                      input_vars[index].retain_data()
  
              if self._output_indexes_to_retain is not None:
                  retained_data = []
                  for index in self._output_indexes_to_retain:
                      ret[index].retain_data()
                      retained_data.append(outputs[index])
                  self._retained_output_data = tuple(retained_data)
  
              self.lazy_grad_sum = configuration.config.lazy_grad_sum
  
      return ret
  
imports: import collections
  import contextlib
  import heapq
  import inspect
  import traceback
  import weakref
  import six
  import chainer
  from chainer import _backprop_utils
  from chainer import backend
  from chainer.backends import cuda
  from chainer import configuration
  from chainer import function_hook
  from chainer.graph_optimizations.static_graph_utilities import static_forward_optimizations
  from chainer import utils
  from chainer.utils import type_check
  from chainer import variable
  import chainerx
label_desc: Use image node_id 7 for calling the FunctionNode obj's underlying member method code with example usage: obj.apply(inputs) and returns: ret, tuple