class_docstr: A function that executes the static schedule of a Chain.
  
  An instance of this class executes the static schedule of computations
  that are equivalent to executing the define-by-run code of a Chain.
  
  This class is used by the `static_graph` decorator to wrap the
  define-by-run
  computations of a chain into two static schedules:
  - The forward schedule corresponds to the computations that are executed by
  the define-by-run code of the `__call__()` method of a chain. The static
  schedule corresponding to these computations can be executed by calling the
  `forward()` method of this class.
  - The backward schedule corresponds to the computations that are executed
  by the sequence of calls to `Function.backward()` that occur during when
  backpropagating the gradients through the same chain. That is, for each
  `Function.forward()` that was called during the forward propagation,
  there will be a corresponding call to `Function.backward()` (of the
  same Function object) in the backward schedule. This backward schedule
  can be executed by calling the `backward()` method of this class.
  
  Note the intended usage of this class:
  
  Recall that a "static chain" referes to a chain that is decorated by the
  `static_graph` decorator.
  During the first forward pass of a static chain, the define-by-run code
  is executed. However,
  for subsequent iterations, that define-by-run code is replaced by an
  instance
  of this function and this function will be called instead. Since the
  static
  schedules contained by this function perform the same computations, it is
  safe (and potentially much more efficient) to simply execute the static
  schedule instead
  of the define-by-run code. See `static_graph` for details.
  
  Args:
      schedule_manager (ScheduleManager): The schedule manager of this
          schedule instance.
      in_vars (tuple of Variable): The flattened tuple of input variables
          that is supplied to
          `__call__()` method of the chain that this schedule corresponds to.
      unique_arrays (list of ndarray): A list of all unique array references
          deeply used in an StaticScheduleFunction instance. It is 'None'
          for the StaticScheduleFunction that corresponds to the "forward"
          schedule, but the contained StaticScheduleFunction for the
          "backward" schedule should take the unique_arrays of the
          "forward" schedule.
class_docstr_tok: ["A","function","that","executes","the","static","schedule","of","a","Chain",".","An","instance","of","this","class","executes","the","static","schedule","of","computations","that","are","equivalent","to","executing","the","define-by-run","code","of","a","Chain",".","This","class","is","used","by","the","`","static_graph","`","decorator","to","wrap","the","define-by-run","computations","of","a","chain","into","two","static","schedules",":","-","The","forward","schedule","corresponds","to","the","computations","that","are","executed","by","the","define-by-run","code","of","the","`","__call__","(",")","`","method","of","a","chain",".","The","static","schedule","corresponding","to","these","computations","can","be","executed","by","calling","the","`","forward","(",")","`","method","of","this","class",".","-","The","backward","schedule","corresponds","to","the","computations","that","are","executed","by","the","sequence","of","calls","to","`","Function.backward","(",")","`","that","occur","during","when","backpropagating","the","gradients","through","the","same","chain",".","That","is",",","for","each","`","Function.forward","(",")","`","that","was","called","during","the","forward","propagation",",","there","will","be","a","corresponding","call","to","`","Function.backward","(",")","`","(","of","the","same","Function","object",")","in","the","backward","schedule",".","This","backward","schedule","can","be","executed","by","calling","the","`","backward","(",")","`","method","of","this","class",".","Note","the","intended","usage","of","this","class",":","Recall","that","a","``","static","chain","''","referes","to","a","chain","that","is","decorated","by","the","`","static_graph","`","decorator",".","During","the","first","forward","pass","of","a","static","chain",",","the","define-by-run","code","is","executed",".","However",",","for","subsequent","iterations",",","that","define-by-run","code","is","replaced","by","an","instance","of","this","function","and","this","function","will","be","called","instead",".","Since","the","static","schedules","contained","by","this","function","perform","the","same","computations",",","it","is","safe","(","and","potentially","much","more","efficient",")","to","simply","execute","the","static","schedule","instead","of","the","define-by-run","code",".","See","`","static_graph","`","for","details",".","Args",":","schedule_manager","(","ScheduleManager",")",":","The","schedule","manager","of","this","schedule","instance",".","in_vars","(","tuple","of","Variable",")",":","The","flattened","tuple","of","input","variables","that","is","supplied","to","`","__call__","(",")","`","method","of","the","chain","that","this","schedule","corresponds","to",".","unique_arrays","(","list","of","ndarray",")",":","A","list","of","all","unique","array","references","deeply","used","in","an","StaticScheduleFunction","instance",".","It","is","'None'","for","the","StaticScheduleFunction","that","corresponds","to","the","``","forward","''","schedule",",","but","the","contained","StaticScheduleFunction","for","the","``","backward","''","schedule","should","take","the","unique_arrays","of","the","``","forward","''","schedule","."]
docstr: Append a function to the static schedule.
  
  Append a function `func` to the static schedule. `func` can
  be any function that is decorated with `@static_code` and that
  was called while executing the static chain's `__call___()`
  method, which contains the define-by-run code. The code
  in the `@static_code` decorator will call this method to
  add the function to the schedule just after it executes in
  the define-by-run code as follows:
  
  `return_arrays = func(*args, **kwargs)`
  
  During the next iteration when the static chain switches from define-
  by-run to the static schedule, a corresponding `ScheduleInfo`
  object will call `func` as above, except that the scheduler might
  make modifications
  to some of the arrays in `kwargs` before and after the function is
  called to implement various memory optimizations.
  
  Args:
      func (function or method): The function to append to the schedule.
          This is a function that was decorated with `@static_code`.
      args: The arguments that were originally supplied to `func` in
          the define-by-run code of the static chain.
      kwargs: The keyword arguments that were originally supplied to
          `func` in the define-by-run code of the static chain.
      func_name (str): Optional name for `func`, for debugging
          purposes.
      return_arrays (tuple of ndarray) or None: The value that is
          returned by `func`, if any.
docstr_tok: ["Append","a","function","to","the","static","schedule",".","Append","a","function","`","func","`","to","the","static","schedule",".","`","func","`","can","be","any","function","that","is","decorated","with","`","@","static_code","`","and","that","was","called","while","executing","the","static","chain","'s","`","__call___","(",")","`","method",",","which","contains","the","define-by-run","code",".","The","code","in","the","`","@","static_code","`","decorator","will","call","this","method","to","add","the","function","to","the","schedule","just","after","it","executes","in","the","define-by-run","code","as","follows",":","`","return_arrays","=","func","(","*","args",",","*","*","kwargs",")","`","During","the","next","iteration","when","the","static","chain","switches","from","define-","by-run","to","the","static","schedule",",","a","corresponding","`","ScheduleInfo","`","object","will","call","`","func","`","as","above",",","except","that","the","scheduler","might","make","modifications","to","some","of","the","arrays","in","`","kwargs","`","before","and","after","the","function","is","called","to","implement","various","memory","optimizations",".","Args",":","func","(","function","or","method",")",":","The","function","to","append","to","the","schedule",".","This","is","a","function","that","was","decorated","with","`","@","static_code","`",".","args",":","The","arguments","that","were","originally","supplied","to","`","func","`","in","the","define-by-run","code","of","the","static","chain",".","kwargs",":","The","keyword","arguments","that","were","originally","supplied","to","`","func","`","in","the","define-by-run","code","of","the","static","chain",".","func_name","(","str",")",":","Optional","name","for","`","func","`",",","for","debugging","purposes",".","return_arrays","(","tuple","of","ndarray",")","or","None",":","The","value","that","is","returned","by","`","func","`",",","if","any","."]
code: def append_function(self, func, args, kwargs, func_name=None):
      """Append a function to the static schedule.
  
      Append a function `func` to the static schedule. `func` can
      be any function that is decorated with `@static_code` and that
      was called while executing the static chain's `__call___()`
      method, which contains the define-by-run code. The code
      in the `@static_code` decorator will call this method to
      add the function to the schedule just after it executes in
      the define-by-run code as follows:
  
      `return_arrays = func(*args, **kwargs)`
  
      During the next iteration when the static chain switches from define-
      by-run to the static schedule, a corresponding `ScheduleInfo`
      object will call `func` as above, except that the scheduler might
      make modifications
      to some of the arrays in `kwargs` before and after the function is
      called to implement various memory optimizations.
  
      Args:
          func (function or method): The function to append to the schedule.
              This is a function that was decorated with `@static_code`.
          args: The arguments that were originally supplied to `func` in
              the define-by-run code of the static chain.
          kwargs: The keyword arguments that were originally supplied to
              `func` in the define-by-run code of the static chain.
          func_name (str): Optional name for `func`, for debugging
              purposes.
          return_arrays (tuple of ndarray) or None: The value that is
              returned by `func`, if any.
  
      """
  
      # Check previous function in the schedule, if available.
      # Check the arrays in the retained inputs/outputs and force them
      # to remain statically allocated in the schedule.
      # ids of any retained arrays.
      retained_ids = set()
  
      last_sched_info_ind = len(self.schedule_info_list) - 1
      if last_sched_info_ind >= 0:
          prev_sched_info = self.schedule_info_list[last_sched_info_ind]
          if prev_sched_info.function_node is not None:
              # get retained inputs/outputs.
              retained_in_vars = (
                  prev_sched_info.function_node.get_retained_inputs()
              )
              retained_out_vars = (
                  prev_sched_info.function_node.get_retained_outputs()
              )
              if (
                  retained_in_vars is not None
                  and retained_out_vars is not None
              ):
                  retained_vars = retained_in_vars + retained_out_vars
              elif retained_in_vars is not None:
                  retained_vars = retained_in_vars
              elif retained_out_vars is not None:
                  retained_vars = retained_out_vars
              else:
                  retained_vars = None
              if retained_vars is not None:
                  for var in retained_vars:
                      retained_ids.add(id(var.data))
  
      for keep_id in retained_ids:
          unique_ind = self.array_id_to_unique_index[keep_id]
          array_info = self.unique_array_infos[unique_ind]
          array_info.retain = True
          # Note: the following line is not actually needed.
          # array_info.array = array_info.weak_ref()
  
      delete_hooks = []
      for unique_ind, ar_info in enumerate(self.unique_array_infos):
          # todo: this is O(N^2) and maybe too slow for large graphs.
          # Optimize it later.
          if ar_info.was_deleted():
              if ar_info.dynamic_deletion_index is None:
                  if self.verbosity_level >= 2:
                      print("Adding delete hook:")
                  delete_hooks.append(unique_ind)
                  ar_info.dynamic_deletion_index = (
                      last_sched_info_ind + 1
                  )
                  ar_info.dynamic_deletion_pass_depth = self.pass_depth
  
      # Call the `@static_code`-decorated function.
      ret = func(*args, **kwargs)
  
      inputs_hooks = []
      if "inputs" in kwargs:
          in_list = kwargs["inputs"]
          assert isinstance(in_list, list)
          for ind, x in enumerate(in_list):
              if _is_xp(x):
                  unique_ind = self.get_unique_index_from_array(x)
                  if unique_ind is None:
                      # Note: we append None here because we cannot store any
                      # additional reference to the array.
                      # Otherwise, it would
                      # prevent garbage collection. Note that a
                      # weak reference
                      # will be stored in the ArrayInfo below.
                      self.unique_arrays.append(None)
                      self.unique_array_infos.append(ArrayInfo(x))
                      unique_ind = len(self.unique_arrays) - 1
                      self.array_id_to_unique_index[id(x)] = unique_ind
                  inputs_hooks.append((ind, unique_ind))
                  # Now that the hook has been added, we can delete
                  # array reference from 'args'.
                  in_list[ind] = None
  
      outputs_hooks = []
      if "outputs" in kwargs:
          out_list = kwargs["outputs"]
          assert isinstance(out_list, list)
          for ind, x in enumerate(out_list):
              if _is_xp(x):
                  unique_ind = self.get_unique_index_from_array(x)
                  if unique_ind is None:
                      self.unique_arrays.append(x)
                      # todo: enable the following line instead once the
                      # auto-intializing hooks are added. This will further
                      # reduce memory usage.
                      # self.unique_arrays.append(None)
                      self.unique_array_infos.append(ArrayInfo(x))
                      unique_ind = len(self.unique_arrays) - 1
                      self.array_id_to_unique_index[id(x)] = unique_ind
                  outputs_hooks.append((ind, unique_ind))
                  # Now that the hook has been added, we can delete
                  # array reference from 'args'.
                  out_list[ind] = None
  
      # A list of hooks (each is a tuple) that will be used to set
      # correct array references in 'unique_arrays' after executing
      # the static schedule function 'func'. These hooks update
      # the references in 'unique_arrays' to refer to the arrays
      # that were dynamically allocated in the return value of
      # 'func'.
      return_hooks = []
      if ret is not None:
          assert isinstance(ret, list) or isinstance(ret, tuple)
          for ret_index, item in enumerate(ret):
              if _is_xp(item):
                  # note: id might not be unique if objects have been
                  # garbage collected.
                  item_id = id(item)
                  unique_index = self.get_unique_index_from_array(item)
                  if unique_index is None:
                      # Note: Append None instead of 'item' to prevent an
                      # extra reference from being stored. Otherwise it
                      # would prevent garbage collection.
                      self.unique_arrays.append(None)
                      ar_info = ArrayInfo(item)
                      ar_info.dynamically_allocated = True
                      sched_info_ind = len(self.schedule_info_list)
                      ar_info.dynamic_allocation_index = sched_info_ind
                      ar_info.dynamic_allocation_pass_depth = (
                          self.pass_depth
                      )
                      self.unique_array_infos.append(ar_info)
                      unique_index = len(self.unique_arrays) - 1
                      self.array_id_to_unique_index[
                          item_id
                      ] = unique_index
                  else:
                      # Since all of the return arrays are supposed to
                      # have been dynamically allocated inside 'func',
                      # they had better not already be in unique_arrays.
                      # If so, it is an error.
                      unique_index = self.array_id_to_unique_index[
                          item_id
                      ]
                      print("the current id: ", item_id)
                      print("the unique_index: ", unique_index)
                      print(
                          "array info: ",
                          self.unique_array_infos[unique_ind],
                      )
                      raise RuntimeError(
                          "Found result array from schedule "
                          "function already in "
                          "unique_arrays!"
                      )
                  return_hooks.append((ret_index, unique_index))
                  self.dynamically_allocated_unique_index.add(
                      unique_index
                  )
  
      if self.verbosity_level >= 2:
          print("Adding function to static schedule: ", func)
  
      self.schedule_info_list.append(
          ScheduleInfo(
              func,
              args,
              kwargs,
              inputs_hooks,
              outputs_hooks,
              return_hooks,
              delete_hooks,
              self.unique_arrays,
              self.unique_array_infos,
              func_name=func_name,
          )
      )
  
      return ret
  
imports: import sys
  import weakref
  import numpy
  import chainer
  from chainer.backends import cuda
  import chainer.function_node
label_desc: Use image node_id 5 for calling the StaticScheduleFunction obj's underlying member method code with example usage: obj.append_function(func, args, kwargs, func_name) and returns: ret