class_docstr: A class implementing image preprocessing for Atari 2600 agents.
  
  Specifically, this provides the following subset from the JAIR paper
  (Bellemare et al., 2013) and Nature DQN paper (Mnih et al., 2015):
  
    * Frame skipping (defaults to 4).
    * Terminal signal when a life is lost (off by default).
    * Grayscale and max-pooling of the last two frames.
    * Downsample the screen to a square image (defaults to 84x84).
  
  More generally, this class follows the preprocessing guidelines set down in
  Machado et al. (2018), "Revisiting the Arcade Learning Environment:
  Evaluation Protocols and Open Problems for General Agents".
  
  It also provides random starting no-ops, which are used in the Rainbow, Apex
  and R2D2 papers.
class_docstr_tok: ["A","class","implementing","image","preprocessing","for","Atari","2600","agents",".","Specifically",",","this","provides","the","following","subset","from","the","JAIR","paper","(","Bellemare","et","al.",",","2013",")","and","Nature","DQN","paper","(","Mnih","et","al.",",","2015",")",":","*","Frame","skipping","(","defaults","to","4",")",".","*","Terminal","signal","when","a","life","is","lost","(","off","by","default",")",".","*","Grayscale","and","max-pooling","of","the","last","two","frames",".","*","Downsample","the","screen","to","a","square","image","(","defaults","to","84x84",")",".","More","generally",",","this","class","follows","the","preprocessing","guidelines","set","down","in","Machado","et","al",".","(","2018",")",",","``","Revisiting","the","Arcade","Learning","Environment",":","Evaluation","Protocols","and","Open","Problems","for","General","Agents","''",".","It","also","provides","random","starting","no-ops",",","which","are","used","in","the","Rainbow",",","Apex","and","R2D2","papers","."]
docstr: Applies the given action in the environment.
  
  Remarks:
  
    * If a terminal state (from life loss or episode end) is reached, this may
      execute fewer than self.frame_skip steps in the environment.
    * Furthermore, in this case the returned observation may not contain valid
      image data and should be ignored.
  
  Args:
    action: The action to be executed.
  
  Returns:
    observation: numpy array, the observation following the action.
    reward: float, the reward following the action.
    is_terminal: bool, whether the environment has reached a terminal state.
      This is true when a life is lost and terminal_on_life_loss, or when the
      episode is over.
    info: Gym API's info data structure.
docstr_tok: ["Applies","the","given","action","in","the","environment",".","Remarks",":","*","If","a","terminal","state","(","from","life","loss","or","episode","end",")","is","reached",",","this","may","execute","fewer","than","self.frame_skip","steps","in","the","environment",".","*","Furthermore",",","in","this","case","the","returned","observation","may","not","contain","valid","image","data","and","should","be","ignored",".","Args",":","action",":","The","action","to","be","executed",".","Returns",":","observation",":","numpy","array",",","the","observation","following","the","action",".","reward",":","float",",","the","reward","following","the","action",".","is_terminal",":","bool",",","whether","the","environment","has","reached","a","terminal","state",".","This","is","true","when","a","life","is","lost","and","terminal_on_life_loss",",","or","when","the","episode","is","over",".","info",":","Gym","API","'s","info","data","structure","."]
code: def step(self, action):
      """Applies the given action in the environment.
  
      Remarks:
  
        * If a terminal state (from life loss or episode end) is reached, this may
          execute fewer than self.frame_skip steps in the environment.
        * Furthermore, in this case the returned observation may not contain valid
          image data and should be ignored.
  
      Args:
        action: The action to be executed.
  
      Returns:
        observation: numpy array, the observation following the action.
        reward: float, the reward following the action.
        is_terminal: bool, whether the environment has reached a terminal state.
          This is true when a life is lost and terminal_on_life_loss, or when the
          episode is over.
        info: Gym API's info data structure.
      """
      accumulated_reward = 0.0
  
      for time_step in range(self.frame_skip):
          # We bypass the Gym observation altogether and directly fetch the
          # grayscale image from the ALE. This is a little faster.
          _, reward, game_over, info = self.environment.step(action)
          accumulated_reward += reward
  
          if self.terminal_on_life_loss:
              new_lives = self.environment.ale.lives()
              is_terminal = game_over or new_lives < self.lives
              self.lives = new_lives
          else:
              is_terminal = game_over
  
          if is_terminal:
              break
          # We max-pool over the last two frames, in grayscale.
          elif time_step >= self.frame_skip - 2:
              t = time_step - (self.frame_skip - 2)
              self._fetch_grayscale_observation(self.screen_buffer[t])
  
      # Pool the last two observations.
      observation = self._pool_and_resize()
  
      self.game_over = game_over
      return observation, accumulated_reward, is_terminal, info
  
file_path: moolib/examples/atari/atari_preprocessing.py
imports: from gym.spaces.box import Box
  import numpy
  import cv2
label_desc: Use image node_id 10 for calling the AtariPreprocessing obj's underlying member method code with example usage: obj.step(action) and returns: observation,  accumulated_reward,  is_terminal,  info