class_docstr: An exploration that adds Ornstein-Uhlenbeck noise to continuous actions.
  
  If explore=True, returns sampled actions plus a noise term X,
  which changes according to this formula:
  Xt+1 = -theta*Xt + sigma*N[0,stddev], where theta, sigma and stddev are
  constants. Also, some completely random period is possible at the
  beginning.
  If explore=False, returns the deterministic action.
class_docstr_tok: ["An","exploration","that","adds","Ornstein-Uhlenbeck","noise","to","continuous","actions",".","If","explore=True",",","returns","sampled","actions","plus","a","noise","term","X",",","which","changes","according","to","this","formula",":","Xt+1","=","-theta","*","Xt","+","sigma","*","N","[","0",",","stddev","]",",","where","theta",",","sigma","and","stddev","are","constants",".","Also",",","some","completely","random","period","is","possible","at","the","beginning",".","If","explore=False",",","returns","the","deterministic","action","."]
docstr: Initializes an Ornstein-Uhlenbeck Exploration object.
  
  Args:
      action_space: The gym action space used by the environment.
      ou_theta: The theta parameter of the Ornstein-Uhlenbeck process.
      ou_sigma: The sigma parameter of the Ornstein-Uhlenbeck process.
      ou_base_scale: A fixed scaling factor, by which all OU-
          noise is multiplied. NOTE: This is on top of the parent
          GaussianNoise's scaling.
      random_timesteps: The number of timesteps for which to act
          completely randomly. Only after this number of timesteps, the
          `self.scale` annealing process will start (see below).
      initial_scale: The initial scaling weight to multiply the
          noise with.
      final_scale: The final scaling weight to multiply the noise with.
      scale_timesteps: The timesteps over which to linearly anneal the
          scaling factor (after(!) having used random actions for
          `random_timesteps` steps.
      scale_schedule: An optional Schedule object to use (instead
          of constructing one from the given parameters).
      framework: One of None, "tf", "torch".
docstr_tok: ["Initializes","an","Ornstein-Uhlenbeck","Exploration","object",".","Args",":","action_space",":","The","gym","action","space","used","by","the","environment",".","ou_theta",":","The","theta","parameter","of","the","Ornstein-Uhlenbeck","process",".","ou_sigma",":","The","sigma","parameter","of","the","Ornstein-Uhlenbeck","process",".","ou_base_scale",":","A","fixed","scaling","factor",",","by","which","all","OU-","noise","is","multiplied",".","NOTE",":","This","is","on","top","of","the","parent","GaussianNoise","'s","scaling",".","random_timesteps",":","The","number","of","timesteps","for","which","to","act","completely","randomly",".","Only","after","this","number","of","timesteps",",","the","`","self.scale","`","annealing","process","will","start","(","see","below",")",".","initial_scale",":","The","initial","scaling","weight","to","multiply","the","noise","with",".","final_scale",":","The","final","scaling","weight","to","multiply","the","noise","with",".","scale_timesteps",":","The","timesteps","over","which","to","linearly","anneal","the","scaling","factor","(","after","(","!",")","having","used","random","actions","for","`","random_timesteps","`","steps",".","scale_schedule",":","An","optional","Schedule","object","to","use","(","instead","of","constructing","one","from","the","given","parameters",")",".","framework",":","One","of","None",",","``","tf","''",",","``","torch","''","."]
code: def __init__(
      self,
      action_space,
      *,
      framework: str,
      ou_theta: float = 0.15,
      ou_sigma: float = 0.2,
      ou_base_scale: float = 0.1,
      random_timesteps: int = 1000,
      initial_scale: float = 1.0,
      final_scale: float = 0.02,
      scale_timesteps: int = 10000,
      scale_schedule: Optional[Schedule] = None,
      **kwargs,
  ):
      """Initializes an Ornstein-Uhlenbeck Exploration object.
  
      Args:
          action_space: The gym action space used by the environment.
          ou_theta: The theta parameter of the Ornstein-Uhlenbeck process.
          ou_sigma: The sigma parameter of the Ornstein-Uhlenbeck process.
          ou_base_scale: A fixed scaling factor, by which all OU-
              noise is multiplied. NOTE: This is on top of the parent
              GaussianNoise's scaling.
          random_timesteps: The number of timesteps for which to act
              completely randomly. Only after this number of timesteps, the
              `self.scale` annealing process will start (see below).
          initial_scale: The initial scaling weight to multiply the
              noise with.
          final_scale: The final scaling weight to multiply the noise with.
          scale_timesteps: The timesteps over which to linearly anneal the
              scaling factor (after(!) having used random actions for
              `random_timesteps` steps.
          scale_schedule: An optional Schedule object to use (instead
              of constructing one from the given parameters).
          framework: One of None, "tf", "torch".
      """
      # The current OU-state value (gets updated each time, an eploration
      # action is computed).
      self.ou_state = get_variable(
          np.array(action_space.low.size * [0.0], dtype=np.float32),
          framework=framework,
          tf_name="ou_state",
          torch_tensor=True,
          device=None,
      )
  
      super().__init__(
          action_space,
          framework=framework,
          random_timesteps=random_timesteps,
          initial_scale=initial_scale,
          final_scale=final_scale,
          scale_timesteps=scale_timesteps,
          scale_schedule=scale_schedule,
          stddev=1.0,  # Force `self.stddev` to 1.0.
          **kwargs,
      )
      self.ou_theta = ou_theta
      self.ou_sigma = ou_sigma
      self.ou_base_scale = ou_base_scale
      # Now that we know the device, move ou_state there, in case of PyTorch.
      if self.framework == "torch" and self.device is not None:
          self.ou_state = self.ou_state.to(self.device)
  
file_path: ray/python/ray/rllib/utils/exploration/ornstein_uhlenbeck_noise.py
imports: import numpy
  from typing import Optional, Union
  from ray.rllib.models.action_dist import ActionDistribution
  from ray.rllib.utils.annotations import override, PublicAPI
  from ray.rllib.utils.exploration.gaussian_noise import GaussianNoise
  from ray.rllib.utils.framework import try_import_tf, try_import_torch, get_variable, TensorType
  from ray.rllib.utils.numpy import convert_to_numpy
  from ray.rllib.utils.schedules import Schedule
  from ray.rllib.utils.tf_utils import zero_logps_from_actions
label_desc: Use image node_id 1 to create a new OrnsteinUhlenbeckNoise object from inherited base classes: GaussianNoise with example: obj = OrnsteinUhlenbeckNoise(action_space)