class_docstr: RL environment created from a generator of sequential data.
  
  This class allows to create RL environments from supervised sequential data,
  such as tokenized natural languague processing tasks. The data comes as:
    (input1, output1, input2, output2, ...)
  where inputs and outputs are all sequences of integers.
  
  For example, with input (2, 3) and output (4, 5), so data = [(2, 3), (4, 5)],
  the sequence of (observations, rewards, actions) will look like:
    2                   = env.reset()                # first observation
    3,    0.0,   _, _   = env.step(ignored_action)
    eos,  0.0,   _, _   = env.step(ignored_action)
    act1, 0.0,   _, _   = env.step(act1)             # observation = action
    act2, 0.0,   _, _   = env.step(act2)             # observation = action
    eos,  score, _, _   = env.step(eos)
  
  where score = metric((4, 5), (act1, act2)) is the reward gotten from
  comparing the two actions to the actual output from the data.
  
  The environment first presents the input as observations, doing this
  sequentially, token-by-token, and ignoring all actions taken by the policy.
  Then, the policy is asked to generate the response, again, token-by-token,
  until it generates EOS. Generated tokens are repeated as observations.
  When EOS is encountered, a metric is computed between the generated
  output and the output from data, and this metric is returned as reward.
class_docstr_tok: ["RL","environment","created","from","a","generator","of","sequential","data",".","This","class","allows","to","create","RL","environments","from","supervised","sequential","data",",","such","as","tokenized","natural","languague","processing","tasks",".","The","data","comes","as",":","(","input1",",","output1",",","input2",",","output2",",","...",")","where","inputs","and","outputs","are","all","sequences","of","integers",".","For","example",",","with","input","(","2",",","3",")","and","output","(","4",",","5",")",",","so","data","=","[","(","2",",","3",")",",","(","4",",","5",")","]",",","the","sequence","of","(","observations",",","rewards",",","actions",")","will","look","like",":","2","=","env.reset","(",")","#","first","observation","3",",","0.0",",","_",",","_","=","env.step","(","ignored_action",")","eos",",","0.0",",","_",",","_","=","env.step","(","ignored_action",")","act1",",","0.0",",","_",",","_","=","env.step","(","act1",")","#","observation","=","action","act2",",","0.0",",","_",",","_","=","env.step","(","act2",")","#","observation","=","action","eos",",","score",",","_",",","_","=","env.step","(","eos",")","where","score","=","metric","(","(","4",",","5",")",",","(","act1",",","act2",")",")","is","the","reward","gotten","from","comparing","the","two","actions","to","the","actual","output","from","the","data",".","The","environment","first","presents","the","input","as","observations",",","doing","this","sequentially",",","token-by-token",",","and","ignoring","all","actions","taken","by","the","policy",".","Then",",","the","policy","is","asked","to","generate","the","response",",","again",",","token-by-token",",","until","it","generates","EOS",".","Generated","tokens","are","repeated","as","observations",".","When","EOS","is","encountered",",","a","metric","is","computed","between","the","generated","output","and","the","output","from","data",",","and","this","metric","is","returned","as","reward","."]
docstr: Reset this environment.
docstr_tok: ["Reset","this","environment","."]
code: def reset(self):
      """Reset this environment."""
      self._cur_sequence = next(self._data)
      # Position contains 2 indices: which sequnece are we in? (input1, output1,
      # input2, output2 and so on) and which token in the sequence are we in?
      self._cur_position = (0, 0)
      self._response = []
      return self.observation
  
file_path: trax/trax/rl/envs/data_envs.py
imports: import gym
  import numpy
label_desc: Use image node_id 6 for calling the SequenceDataEnv obj's underlying member method code with example usage: obj.reset() and returns: self