class_docstr: Interface for environments with sub-step structure, which is generally enough to cover multi-step,
  hierarchical and multi-agent environments.
  
  This environment can continuously create and destroy a previously unknown,
  unlimited number of actors during the course of an episode. Every actor is associated with one of the
  available policies.
  
  The lifecycle of the environment is decoupled from the lifecycle of the actors. The interaction loop should
  continue, until the environment as a whole is set to done, which is returned as usual by the step() function.
  Individual actors might end earlier, which can be queried by the is_actor_done() method.
  
  Pseudo-code of the interaction loop:
  
  # start a new episode
  observation = env.reset()
  
  while not done:
      # find out which actor is next to act (dictated by the env)
      sub_step_key, actor_id = env.actor_id()
  
      # obtain the next action from the policy
      action = sample_from_policy(observation, sub_step_key, actor_id)
  
      # step the env
      observation, reward, done, info = env.step(action)
  
      # optionally use is_actor_done() to find out if the actor was terminated (relevant during training)
class_docstr_tok: ["Interface","for","environments","with","sub-step","structure",",","which","is","generally","enough","to","cover","multi-step",",","hierarchical","and","multi-agent","environments",".","This","environment","can","continuously","create","and","destroy","a","previously","unknown",",","unlimited","number","of","actors","during","the","course","of","an","episode",".","Every","actor","is","associated","with","one","of","the","available","policies",".","The","lifecycle","of","the","environment","is","decoupled","from","the","lifecycle","of","the","actors",".","The","interaction","loop","should","continue",",","until","the","environment","as","a","whole","is","set","to","done",",","which","is","returned","as","usual","by","the","step","(",")","function",".","Individual","actors","might","end","earlier",",","which","can","be","queried","by","the","is_actor_done","(",")","method",".","Pseudo-code","of","the","interaction","loop",":","#","start","a","new","episode","observation","=","env.reset","(",")","while","not","done",":","#","find","out","which","actor","is","next","to","act","(","dictated","by","the","env",")","sub_step_key",",","actor_id","=","env.actor_id","(",")","#","obtain","the","next","action","from","the","policy","action","=","sample_from_policy","(","observation",",","sub_step_key",",","actor_id",")","#","step","the","env","observation",",","reward",",","done",",","info","=","env.step","(","action",")","#","optionally","use","is_actor_done","(",")","to","find","out","if","the","actor","was","terminated","(","relevant","during","training",")"]
docstr: Optional. If this is a multi-step or multi-agent environment, this method should return
  the last reward for all actors from the last structured step.
  
  I.e., the length of the returned array should be equal to the number of sub-steps in the last structured
  step.
  
  This is useful e.g. for multi-agent cases, where we first collect actions from all agents, and then
  process them together and calculate rewards for all agents. In such scenario, the reward of individual
  agents is not know until after actions of all agents have been collected. Then, individual
  sub-steps might return None as agent reward, and then the reward for all agents can be queried
  and redistributed once the structured step is done.
  
  :return: If available: Array of rewards, one per each actor in the last structured step. None otherwise.
docstr_tok: ["Optional",".","If","this","is","a","multi-step","or","multi-agent","environment",",","this","method","should","return","the","last","reward","for","all","actors","from","the","last","structured","step",".","I.e.",",","the","length","of","the","returned","array","should","be","equal","to","the","number","of","sub-steps","in","the","last","structured","step",".","This","is","useful","e.g",".","for","multi-agent","cases",",","where","we","first","collect","actions","from","all","agents",",","and","then","process","them","together","and","calculate","rewards","for","all","agents",".","In","such","scenario",",","the","reward","of","individual","agents","is","not","know","until","after","actions","of","all","agents","have","been","collected",".","Then",",","individual","sub-steps","might","return","None","as","agent","reward",",","and","then","the","reward","for","all","agents","can","be","queried","and","redistributed","once","the","structured","step","is","done",".",":","return",":","If","available",":","Array","of","rewards",",","one","per","each","actor","in","the","last","structured","step",".","None","otherwise","."]
code: def get_actor_rewards(self) -> Optional[np.ndarray]:
      """Optional. If this is a multi-step or multi-agent environment, this method should return
      the last reward for all actors from the last structured step.
  
      I.e., the length of the returned array should be equal to the number of sub-steps in the last structured
      step.
  
      This is useful e.g. for multi-agent cases, where we first collect actions from all agents, and then
      process them together and calculate rewards for all agents. In such scenario, the reward of individual
      agents is not know until after actions of all agents have been collected. Then, individual
      sub-steps might return None as agent reward, and then the reward for all agents can be queried
      and redistributed once the structured step is done.
  
      :return: If available: Array of rewards, one per each actor in the last structured step. None otherwise.
      """
      return None
  
file_path: maze/maze/core/env/structured_env.py
imports: from abc import abstractmethod
  from typing import Union, Dict, NamedTuple, Optional
  import numpy
  from maze.core.env.base_env import BaseEnv
label_desc: Use image node_id 4 for calling the StructuredEnv obj's underlying member method code with example usage: obj.get_actor_rewards() and returns: None