class_docstr: RL environment created from a generator of sequential data.
  
  This class allows to create RL environments from supervised sequential data,
  such as tokenized natural languague processing tasks. The data comes as:
    (input1, output1, input2, output2, ...)
  where inputs and outputs are all sequences of integers.
  
  For example, with input (2, 3) and output (4, 5), so data = [(2, 3), (4, 5)],
  the sequence of (observations, rewards, actions) will look like:
    2                   = env.reset()                # first observation
    3,    0.0,   _, _   = env.step(ignored_action)
    eos,  0.0,   _, _   = env.step(ignored_action)
    act1, 0.0,   _, _   = env.step(act1)             # observation = action
    act2, 0.0,   _, _   = env.step(act2)             # observation = action
    eos,  score, _, _   = env.step(eos)
  
  where score = metric((4, 5), (act1, act2)) is the reward gotten from
  comparing the two actions to the actual output from the data.
  
  The environment first presents the input as observations, doing this
  sequentially, token-by-token, and ignoring all actions taken by the policy.
  Then, the policy is asked to generate the response, again, token-by-token,
  until it generates EOS. Generated tokens are repeated as observations.
  When EOS is encountered, a metric is computed between the generated
  output and the output from data, and this metric is returned as reward.
class_docstr_tok: ["RL","environment","created","from","a","generator","of","sequential","data",".","This","class","allows","to","create","RL","environments","from","supervised","sequential","data",",","such","as","tokenized","natural","languague","processing","tasks",".","The","data","comes","as",":","(","input1",",","output1",",","input2",",","output2",",","...",")","where","inputs","and","outputs","are","all","sequences","of","integers",".","For","example",",","with","input","(","2",",","3",")","and","output","(","4",",","5",")",",","so","data","=","[","(","2",",","3",")",",","(","4",",","5",")","]",",","the","sequence","of","(","observations",",","rewards",",","actions",")","will","look","like",":","2","=","env.reset","(",")","#","first","observation","3",",","0.0",",","_",",","_","=","env.step","(","ignored_action",")","eos",",","0.0",",","_",",","_","=","env.step","(","ignored_action",")","act1",",","0.0",",","_",",","_","=","env.step","(","act1",")","#","observation","=","action","act2",",","0.0",",","_",",","_","=","env.step","(","act2",")","#","observation","=","action","eos",",","score",",","_",",","_","=","env.step","(","eos",")","where","score","=","metric","(","(","4",",","5",")",",","(","act1",",","act2",")",")","is","the","reward","gotten","from","comparing","the","two","actions","to","the","actual","output","from","the","data",".","The","environment","first","presents","the","input","as","observations",",","doing","this","sequentially",",","token-by-token",",","and","ignoring","all","actions","taken","by","the","policy",".","Then",",","the","policy","is","asked","to","generate","the","response",",","again",",","token-by-token",",","until","it","generates","EOS",".","Generated","tokens","are","repeated","as","observations",".","When","EOS","is","encountered",",","a","metric","is","computed","between","the","generated","output","and","the","output","from","data",",","and","this","metric","is","returned","as","reward","."]
docstr: The constructor.
  
  Args:
    data_stream: A python generator creating lists or tuples of
      sequences (list, tuples or numpy arrays) of integers.
    vocab_size: Integer, the size of the vocabulary. All integers in the
      data stream must be positive and smaller than this value.
    metric: A function taking two lists of integers and returning a float.
      If None, we use per-token accuracy as the default metric.
    eos_id: Integer, the id of the EOS symbol.
    max_length: Integer, maximum length of the policy reply to avoid
      infinite episodes if policy never produces EOS.
  
  Returns:
    A new environment which presents the data and compares the policy
    response with the expected data, returning metric as reward.
docstr_tok: ["The","constructor",".","Args",":","data_stream",":","A","python","generator","creating","lists","or","tuples","of","sequences","(","list",",","tuples","or","numpy","arrays",")","of","integers",".","vocab_size",":","Integer",",","the","size","of","the","vocabulary",".","All","integers","in","the","data","stream","must","be","positive","and","smaller","than","this","value",".","metric",":","A","function","taking","two","lists","of","integers","and","returning","a","float",".","If","None",",","we","use","per-token","accuracy","as","the","default","metric",".","eos_id",":","Integer",",","the","id","of","the","EOS","symbol",".","max_length",":","Integer",",","maximum","length","of","the","policy","reply","to","avoid","infinite","episodes","if","policy","never","produces","EOS",".","Returns",":","A","new","environment","which","presents","the","data","and","compares","the","policy","response","with","the","expected","data",",","returning","metric","as","reward","."]
code: def __init__(
      self,
      data_stream,
      vocab_size,
      metric=None,
      eos_id=1,
      max_length=1000,
  ):
      """The constructor.
  
      Args:
        data_stream: A python generator creating lists or tuples of
          sequences (list, tuples or numpy arrays) of integers.
        vocab_size: Integer, the size of the vocabulary. All integers in the
          data stream must be positive and smaller than this value.
        metric: A function taking two lists of integers and returning a float.
          If None, we use per-token accuracy as the default metric.
        eos_id: Integer, the id of the EOS symbol.
        max_length: Integer, maximum length of the policy reply to avoid
          infinite episodes if policy never produces EOS.
  
      Returns:
        A new environment which presents the data and compares the policy
        response with the expected data, returning metric as reward.
      """
      self._data = data_stream
      self._vocab_size = vocab_size
      self._eos = eos_id
      self._max_length = max_length
      self._metric = _accuracy if metric is None else metric
      self.reset()
  
file_path: trax/trax/rl/envs/data_envs.py
imports: import gym
  import numpy
label_desc: Use image node_id 1 to create a new SequenceDataEnv object from inherited base classes: object with example: obj = SequenceDataEnv(data_stream, vocab_size, metric, eos_id, max_length)