class_docstr: Training helper that restores from checkpoint and creates session.
  
  This class is a small wrapper that takes care of session creation and
  checkpoint recovery. It also provides functions that to facilitate
  coordination among multiple training threads or processes.
  
  * Checkpointing trained variables as the training progresses.
  * Initializing variables on startup, restoring them from the most recent
    checkpoint after a crash, or wait for checkpoints to become available.
  
  ### Usage:
  
  ```python
  with tf.Graph().as_default():
     ...add operations to the graph...
    # Create a SessionManager that will checkpoint the model in '/tmp/mydir'.
    sm = SessionManager()
    sess = sm.prepare_session(master, init_op, saver, checkpoint_dir)
    # Use the session to train the graph.
    while True:
      sess.run(<my_train_op>)
  ```
  
  `prepare_session()` initializes or restores a model. It requires `init_op`
  and `saver` as an argument.
  
  A second process could wait for the model to be ready by doing the following:
  
  ```python
  with tf.Graph().as_default():
     ...add operations to the graph...
    # Create a SessionManager that will wait for the model to become ready.
    sm = SessionManager()
    sess = sm.wait_for_session(master)
    # Use the session to train the graph.
    while True:
      sess.run(<my_train_op>)
  ```
  
  `wait_for_session()` waits for a model to be initialized by other processes.
class_docstr_tok: ["Training","helper","that","restores","from","checkpoint","and","creates","session",".","This","class","is","a","small","wrapper","that","takes","care","of","session","creation","and","checkpoint","recovery",".","It","also","provides","functions","that","to","facilitate","coordination","among","multiple","training","threads","or","processes",".","*","Checkpointing","trained","variables","as","the","training","progresses",".","*","Initializing","variables","on","startup",",","restoring","them","from","the","most","recent","checkpoint","after","a","crash",",","or","wait","for","checkpoints","to","become","available",".","#","#","#","Usage",":","``","`","python","with","tf.Graph","(",")",".as_default","(",")",":","...","add","operations","to","the","graph","...","#","Create","a","SessionManager","that","will","checkpoint","the","model","in","'\/tmp\/mydir","'",".","sm","=","SessionManager","(",")","sess","=","sm.prepare_session","(","master",",","init_op",",","saver",",","checkpoint_dir",")","#","Use","the","session","to","train","the","graph",".","while","True",":","sess.run","(","<","my_train_op",">",")","``","`","`","prepare_session","(",")","`","initializes","or","restores","a","model",".","It","requires","`","init_op","`","and","`","saver","`","as","an","argument",".","A","second","process","could","wait","for","the","model","to","be","ready","by","doing","the","following",":","``","`","python","with","tf.Graph","(",")",".as_default","(",")",":","...","add","operations","to","the","graph","...","#","Create","a","SessionManager","that","will","wait","for","the","model","to","become","ready",".","sm","=","SessionManager","(",")","sess","=","sm.wait_for_session","(","master",")","#","Use","the","session","to","train","the","graph",".","while","True",":","sess.run","(","<","my_train_op",">",")","``","`","`","wait_for_session","(",")","`","waits","for","a","model","to","be","initialized","by","other","processes","."]
docstr: Creates a SessionManager.
  
  The `local_init_op` is an `Operation` that is run always after a new session
  was created. If `None`, this step is skipped.
  
  The `ready_op` is an `Operation` used to check if the model is ready.  The
  model is considered ready if that operation returns an empty 1D string
  tensor. If the operation returns a non empty 1D string tensor, the elements
  are concatenated and used to indicate to the user why the model is not
  ready.
  
  The `ready_for_local_init_op` is an `Operation` used to check if the model
  is ready to run local_init_op.  The model is considered ready if that
  operation returns an empty 1D string tensor. If the operation returns a non
  empty 1D string tensor, the elements are concatenated and used to indicate
  to the user why the model is not ready.
  
  If `ready_op` is `None`, the model is not checked for readiness.
  
  `recovery_wait_secs` is the number of seconds between checks that
  the model is ready.  It is used by processes to wait for a model to
  be initialized or restored.  Defaults to 30 seconds.
  
  Args:
    local_init_op: An `Operation` run immediately after session creation.
       Usually used to initialize tables and local variables.
    ready_op: An `Operation` to check if the model is initialized.
    ready_for_local_init_op: An `Operation` to check if the model is ready
       to run local_init_op.
    graph: The `Graph` that the model will use.
    recovery_wait_secs: Seconds between checks for the model to be ready.
    local_init_run_options: RunOptions to be passed to session.run when
      executing the local_init_op.
    local_init_feed_dict: Optional session feed dictionary to use when running
      the local_init_op.
  
  Raises:
    ValueError: If ready_for_local_init_op is not None but local_init_op is
      None
docstr_tok: ["Creates","a","SessionManager",".","The","`","local_init_op","`","is","an","`","Operation","`","that","is","run","always","after","a","new","session","was","created",".","If","`","None","`",",","this","step","is","skipped",".","The","`","ready_op","`","is","an","`","Operation","`","used","to","check","if","the","model","is","ready",".","The","model","is","considered","ready","if","that","operation","returns","an","empty","1D","string","tensor",".","If","the","operation","returns","a","non","empty","1D","string","tensor",",","the","elements","are","concatenated","and","used","to","indicate","to","the","user","why","the","model","is","not","ready",".","The","`","ready_for_local_init_op","`","is","an","`","Operation","`","used","to","check","if","the","model","is","ready","to","run","local_init_op",".","The","model","is","considered","ready","if","that","operation","returns","an","empty","1D","string","tensor",".","If","the","operation","returns","a","non","empty","1D","string","tensor",",","the","elements","are","concatenated","and","used","to","indicate","to","the","user","why","the","model","is","not","ready",".","If","`","ready_op","`","is","`","None","`",",","the","model","is","not","checked","for","readiness",".","`","recovery_wait_secs","`","is","the","number","of","seconds","between","checks","that","the","model","is","ready",".","It","is","used","by","processes","to","wait","for","a","model","to","be","initialized","or","restored",".","Defaults","to","30","seconds",".","Args",":","local_init_op",":","An","`","Operation","`","run","immediately","after","session","creation",".","Usually","used","to","initialize","tables","and","local","variables",".","ready_op",":","An","`","Operation","`","to","check","if","the","model","is","initialized",".","ready_for_local_init_op",":","An","`","Operation","`","to","check","if","the","model","is","ready","to","run","local_init_op",".","graph",":","The","`","Graph","`","that","the","model","will","use",".","recovery_wait_secs",":","Seconds","between","checks","for","the","model","to","be","ready",".","local_init_run_options",":","RunOptions","to","be","passed","to","session.run","when","executing","the","local_init_op",".","local_init_feed_dict",":","Optional","session","feed","dictionary","to","use","when","running","the","local_init_op",".","Raises",":","ValueError",":","If","ready_for_local_init_op","is","not","None","but","local_init_op","is","None"]
code: def __init__(
      self,
      local_init_op: ops.Operation = None,
      ready_op: ops.Operation = None,
      ready_for_local_init_op: ops.Operation = None,
      graph: ops.Graph = None,
      recovery_wait_secs=30,
      local_init_run_options: "distribute_lib.RunOptions" = None,
      local_init_feed_dict=None,
  ):
      """Creates a SessionManager.
  
      The `local_init_op` is an `Operation` that is run always after a new session
      was created. If `None`, this step is skipped.
  
      The `ready_op` is an `Operation` used to check if the model is ready.  The
      model is considered ready if that operation returns an empty 1D string
      tensor. If the operation returns a non empty 1D string tensor, the elements
      are concatenated and used to indicate to the user why the model is not
      ready.
  
      The `ready_for_local_init_op` is an `Operation` used to check if the model
      is ready to run local_init_op.  The model is considered ready if that
      operation returns an empty 1D string tensor. If the operation returns a non
      empty 1D string tensor, the elements are concatenated and used to indicate
      to the user why the model is not ready.
  
      If `ready_op` is `None`, the model is not checked for readiness.
  
      `recovery_wait_secs` is the number of seconds between checks that
      the model is ready.  It is used by processes to wait for a model to
      be initialized or restored.  Defaults to 30 seconds.
  
      Args:
        local_init_op: An `Operation` run immediately after session creation.
           Usually used to initialize tables and local variables.
        ready_op: An `Operation` to check if the model is initialized.
        ready_for_local_init_op: An `Operation` to check if the model is ready
           to run local_init_op.
        graph: The `Graph` that the model will use.
        recovery_wait_secs: Seconds between checks for the model to be ready.
        local_init_run_options: RunOptions to be passed to session.run when
          executing the local_init_op.
        local_init_feed_dict: Optional session feed dictionary to use when running
          the local_init_op.
  
      Raises:
        ValueError: If ready_for_local_init_op is not None but local_init_op is
          None
      """
      # Sets default values of arguments.
      if graph is None:
          graph = ops.get_default_graph()
      self._local_init_op = local_init_op
      self._ready_op = ready_op
      self._ready_for_local_init_op = ready_for_local_init_op
      self._graph = graph
      self._recovery_wait_secs = recovery_wait_secs
      self._target = None
      self._local_init_run_options = local_init_run_options
      self._local_init_feed_dict = local_init_feed_dict
      if ready_for_local_init_op is not None and local_init_op is None:
          raise ValueError(
              "If you pass a ready_for_local_init_op "
              "you must also pass a local_init_op "
              ", ready_for_local_init_op [%s]" % ready_for_local_init_op
          )
  
imports: import time
  from typing import Optional, Tuple
  import numpy
  from tensorflow.python.checkpoint import checkpoint_management
  from tensorflow.python.client import session
  from tensorflow.python.distribute import distribute_lib
  from tensorflow.python.framework import errors
  from tensorflow.python.framework import ops
  from tensorflow.python.platform import tf_logging
  from tensorflow.python.training import saver
  from tensorflow.python.util.tf_export import tf_export
label_desc: Use image node_id 1 to create a new SessionManager object  with example: obj = SessionManager(local_init_op, ready_op, ready_for_local_init_op, graph, recovery_wait_secs, local_init_run_options, local_init_feed_dict)