class_docstr: Options for cross device communications like All-reduce.
  
  This can be passed to methods like
  `tf.distribute.get_replica_context().all_reduce()` to optimize collective
  operation performance. Note that these are only hints, which may or may not
  change the actual behavior. Some options only apply to certain strategy and
  are ignored by others.
  
  One common optimization is to break gradients all-reduce into multiple packs
  so that weight updates can overlap with gradient all-reduce.
  
  Examples:
  
  ```python
  options = tf.distribute.experimental.CommunicationOptions(
      bytes_per_pack=50 * 1024 * 1024,
      timeout_seconds=120.0,
      implementation=tf.distribute.experimental.CommunicationImplementation.NCCL
  )
  grads = tf.distribute.get_replica_context().all_reduce(
      'sum', grads, options=options)
  optimizer.apply_gradients(zip(grads, vars),
      experimental_aggregate_gradients=False)
  ```
class_docstr_tok: ["Options","for","cross","device","communications","like","All-reduce",".","This","can","be","passed","to","methods","like","`","tf.distribute.get_replica_context","(",")",".all_reduce","(",")","`","to","optimize","collective","operation","performance",".","Note","that","these","are","only","hints",",","which","may","or","may","not","change","the","actual","behavior",".","Some","options","only","apply","to","certain","strategy","and","are","ignored","by","others",".","One","common","optimization","is","to","break","gradients","all-reduce","into","multiple","packs","so","that","weight","updates","can","overlap","with","gradient","all-reduce",".","Examples",":","``","`","python","options","=","tf.distribute.experimental.CommunicationOptions","(","bytes_per_pack=50","*","1024","*","1024",",","timeout_seconds=120.0",",","implementation=tf.distribute.experimental.CommunicationImplementation.NCCL",")","grads","=","tf.distribute.get_replica_context","(",")",".all_reduce","(","'sum","'",",","grads",",","options=options",")","optimizer.apply_gradients","(","zip","(","grads",",","vars",")",",","experimental_aggregate_gradients=False",")","``","`"]
docstr: Creates a CollectiveHints.
  
  Args:
    bytes_per_pack: a non-negative integer. Breaks collective operations into
      packs of certain size. If it's zero, the value is determined
      automatically. This hint is respected by all multi-replica strategies
      except `TPUStrategy`.
    timeout_seconds: a float or None, timeout in seconds. If not None, the
      collective raises `tf.errors.DeadlineExceededError` if it takes longer
      than this timeout. Zero disables timeout. This can be useful when
      debugging hanging issues.  This should only be used for debugging since
      it creates a new thread for each collective, i.e. an overhead of
      `timeout_seconds * num_collectives_per_second` more threads. This only
      works for `tf.distribute.experimental.MultiWorkerMirroredStrategy`.
    implementation: a
      `tf.distribute.experimental.CommunicationImplementation`. This is a hint
      on the preferred communication implementation. Possible values include
      `AUTO`, `RING`, and `NCCL`. NCCL is generally more performant for GPU,
      but doesn't work for CPU. This only works for
      `tf.distribute.experimental.MultiWorkerMirroredStrategy`.
  
  Raises:
    ValueError: When arguments have invalid value.
docstr_tok: ["Creates","a","CollectiveHints",".","Args",":","bytes_per_pack",":","a","non-negative","integer",".","Breaks","collective","operations","into","packs","of","certain","size",".","If","it","'s","zero",",","the","value","is","determined","automatically",".","This","hint","is","respected","by","all","multi-replica","strategies","except","`","TPUStrategy","`",".","timeout_seconds",":","a","float","or","None",",","timeout","in","seconds",".","If","not","None",",","the","collective","raises","`","tf.errors.DeadlineExceededError","`","if","it","takes","longer","than","this","timeout",".","Zero","disables","timeout",".","This","can","be","useful","when","debugging","hanging","issues",".","This","should","only","be","used","for","debugging","since","it","creates","a","new","thread","for","each","collective",",","i.e",".","an","overhead","of","`","timeout_seconds","*","num_collectives_per_second","`","more","threads",".","This","only","works","for","`","tf.distribute.experimental.MultiWorkerMirroredStrategy","`",".","implementation",":","a","`","tf.distribute.experimental.CommunicationImplementation","`",".","This","is","a","hint","on","the","preferred","communication","implementation",".","Possible","values","include","`","AUTO","`",",","`","RING","`",",","and","`","NCCL","`",".","NCCL","is","generally","more","performant","for","GPU",",","but","does","n't","work","for","CPU",".","This","only","works","for","`","tf.distribute.experimental.MultiWorkerMirroredStrategy","`",".","Raises",":","ValueError",":","When","arguments","have","invalid","value","."]
code: def __init__(
      self,
      bytes_per_pack=0,
      timeout_seconds=None,
      implementation=CommunicationImplementation.AUTO,
  ):
      """Creates a CollectiveHints.
  
      Args:
        bytes_per_pack: a non-negative integer. Breaks collective operations into
          packs of certain size. If it's zero, the value is determined
          automatically. This hint is respected by all multi-replica strategies
          except `TPUStrategy`.
        timeout_seconds: a float or None, timeout in seconds. If not None, the
          collective raises `tf.errors.DeadlineExceededError` if it takes longer
          than this timeout. Zero disables timeout. This can be useful when
          debugging hanging issues.  This should only be used for debugging since
          it creates a new thread for each collective, i.e. an overhead of
          `timeout_seconds * num_collectives_per_second` more threads. This only
          works for `tf.distribute.experimental.MultiWorkerMirroredStrategy`.
        implementation: a
          `tf.distribute.experimental.CommunicationImplementation`. This is a hint
          on the preferred communication implementation. Possible values include
          `AUTO`, `RING`, and `NCCL`. NCCL is generally more performant for GPU,
          but doesn't work for CPU. This only works for
          `tf.distribute.experimental.MultiWorkerMirroredStrategy`.
  
      Raises:
        ValueError: When arguments have invalid value.
      """
      pass
  
file_path: tensorflow-upstream/tensorflow/python/distribute/collective_util.py
imports: import copy
  import enum
  from tensorflow.python.util import deprecation
  from tensorflow.python.util.tf_export import tf_export
label_desc: Use image node_id 2 to create a new _OptionsExported object from inherited base classes: object with example: obj = _OptionsExported(bytes_per_pack, timeout_seconds, implementation)