class_docstr: Hints for collective operations like AllReduce.
  
  This can be passed to methods like
  `tf.distribute.get_replica_context().all_reduce()` to optimize collective
  operation performance. Note that these are only hints, which may or may not
  change the actual behavior. Some options only apply to certain strategy and
  are ignored by others.
  
  One common optimization is to break gradients all-reduce into multiple packs
  so that weight updates can overlap with gradient all-reduce.
  
  Examples:
  
  - bytes_per_pack
  
  ```python
  hints = tf.distribute.experimental.CollectiveHints(
      bytes_per_pack=50 * 1024 * 1024)
  grads = tf.distribute.get_replica_context().all_reduce(
      'sum', grads, experimental_hints=hints)
  optimizer.apply_gradients(zip(grads, vars),
      experimental_aggregate_gradients=False)
  ```
  
  - timeout_seconds
  
  ```python
  strategy = tf.distribute.MirroredStrategy()
  hints = tf.distribute.experimental.CollectiveHints(
      timeout_seconds=120.0)
  try:
    strategy.reduce("sum", v, axis=None, experimental_hints=hints)
  except tf.errors.DeadlineExceededError:
    do_something()
  ```
class_docstr_tok: ["Hints","for","collective","operations","like","AllReduce",".","This","can","be","passed","to","methods","like","`","tf.distribute.get_replica_context","(",")",".all_reduce","(",")","`","to","optimize","collective","operation","performance",".","Note","that","these","are","only","hints",",","which","may","or","may","not","change","the","actual","behavior",".","Some","options","only","apply","to","certain","strategy","and","are","ignored","by","others",".","One","common","optimization","is","to","break","gradients","all-reduce","into","multiple","packs","so","that","weight","updates","can","overlap","with","gradient","all-reduce",".","Examples",":","-","bytes_per_pack","``","`","python","hints","=","tf.distribute.experimental.CollectiveHints","(","bytes_per_pack=50","*","1024","*","1024",")","grads","=","tf.distribute.get_replica_context","(",")",".all_reduce","(","'sum","'",",","grads",",","experimental_hints=hints",")","optimizer.apply_gradients","(","zip","(","grads",",","vars",")",",","experimental_aggregate_gradients=False",")","``","`","-","timeout_seconds","``","`","python","strategy","=","tf.distribute.MirroredStrategy","(",")","hints","=","tf.distribute.experimental.CollectiveHints","(","timeout_seconds=120.0",")","try",":","strategy.reduce","(","``","sum","''",",","v",",","axis=None",",","experimental_hints=hints",")","except","tf.errors.DeadlineExceededError",":","do_something","(",")","``","`"]
code: def __new__(cls, bytes_per_pack=0, timeout_seconds=None):
      return Options(
          bytes_per_pack=bytes_per_pack, timeout_seconds=timeout_seconds
      )
  
file_path: tensorflow-upstream/tensorflow/python/distribute/collective_util.py
imports: import copy
  import enum
  from tensorflow.python.util import deprecation
  from tensorflow.python.util.tf_export import tf_export
label_desc: Use image node_id 1 for calling the Hints obj's underlying member method code with example usage: obj.__new__(cls, bytes_per_pack, timeout_seconds) and returns: Options