class_docstr: Implements sequence to sequence translation models.
  
  The model is based on the description in Sutskever et al., "Sequence to
  Sequence Learning with Neural Networks" (https://arxiv.org/abs/1409.3215),
  although this implementation uses GRUs instead of LSTMs. The goal is to
  take sequences of tokens as input, and translate each one into a different
  output sequence. The input and output sequences can both be of variable
  length, and an output sequence need not have the same length as the input
  sequence it was generated from. For example, these models were originally
  developed for use in natural language processing. In that context, the
  input might be a sequence of English words, and the output might be a
  sequence of French words. The goal would be to train the model to translate
  sentences from English to French.
  
  The model consists of two parts called the "encoder" and "decoder". Each one
  consists of a stack of recurrent layers. The job of the encoder is to
  transform the input sequence into a single, fixed length vector called the
  "embedding". That vector contains all relevant information from the input
  sequence. The decoder then transforms the embedding vector into the output
  sequence.
  
  These models can be used for various purposes. First and most obviously,
  they can be used for sequence to sequence translation. In any case where you
  have sequences of tokens, and you want to translate each one into a different
  sequence, a SeqToSeq model can be trained to perform the translation.
  
  Another possible use case is transforming variable length sequences into
  fixed length vectors. Many types of models require their inputs to have a
  fixed shape, which makes it difficult to use them with variable sized inputs
  (for example, when the input is a molecule, and different molecules have
  different numbers of atoms). In that case, you can train a SeqToSeq model as
  an autoencoder, so that it tries to make the output sequence identical to the
  input one. That forces the embedding vector to contain all information from
  the original sequence. You can then use the encoder for transforming
  sequences into fixed length embedding vectors, suitable to use as inputs to
  other types of models.
  
  Another use case is to train the decoder for use as a generative model. Here
  again you begin by training the SeqToSeq model as an autoencoder. Once
  training is complete, you can supply arbitrary embedding vectors, and
  transform each one into an output sequence. When used in this way, you
  typically train it as a variational autoencoder. This adds random noise to
  the encoder, and also adds a constraint term to the loss that forces the
  embedding vector to have a unit Gaussian distribution. You can then pick
  random vectors from a Gaussian distribution, and the output sequences should
  follow the same distribution as the training data.
  
  When training as a variational autoencoder, it is best to use KL cost
  annealing, as described in https://arxiv.org/abs/1511.06349. The constraint
  term in the loss is initially set to 0, so the optimizer just tries to
  minimize the reconstruction loss. Once it has made reasonable progress
  toward that, the constraint term can be gradually turned back on. The range
  of steps over which this happens is configurable.
  
  In this class, we establish a sequential model for the Sequence to Sequence (DTNN) [1]_.
  
  Examples
  --------
  >>> import torch
  >>> from deepchem.models.torch_models.seqtoseq import SeqToSeqModel
  >>> data = [
  ...     ("Cc1cccc(N2CCN(C(=O)C34CC5CC(CC(C5)C3)C4)CC2)c1C",
  ...      "Cc1cccc(N2CCN(C(=O)C34CC5CC(CC(C5)C3)C4)CC2)c1C"),
  ...     ("Cn1ccnc1SCC(=O)Nc1ccc(Oc2ccccc2)cc1",
  ...      "Cn1ccnc1SCC(=O)Nc1ccc(Oc2ccccc2)cc1"),
  ...     ("COc1cc2c(cc1NC(=O)CN1C(=O)NC3(CCc4ccccc43)C1=O)oc1ccccc12",
  ...      "COc1cc2c(cc1NC(=O)CN1C(=O)NC3(CCc4ccccc43)C1=O)oc1ccccc12"),
  ...     ("O=C1/C(=C/NC2CCS(=O)(=O)C2)c2ccccc2C(=O)N1c1ccccc1",
  ...      "O=C1/C(=C/NC2CCS(=O)(=O)C2)c2ccccc2C(=O)N1c1ccccc1"),
  ...     ("NC(=O)NC(Cc1ccccc1)C(=O)O",
  ...     "NC(=O)NC(Cc1ccccc1)C(=O)O")]
  >>> train_smiles = [s[0] for s in data]
  >>> tokens = set()
  >>> for s in train_smiles:
  ...     tokens = tokens.union(set(c for c in s))
  >>> tokens = sorted(list(tokens))
  >>> from deepchem.models.optimizers import Adam, ExponentialDecay
  >>> max_length = max(len(s) for s in train_smiles)
  >>> batch_size = 100
  >>> batches_per_epoch = len(train_smiles) / batch_size
  >>> model = SeqToSeqModel(
  ...     tokens,
  ...     tokens,
  ...     max_length,
  ...     encoder_layers=2,
  ...     decoder_layers=2,
  ...     embedding_dimension=256,
  ...     model_dir="fingerprint",
  ...     batch_size=batch_size,
  ...     learning_rate=ExponentialDecay(0.001, 0.9, batches_per_epoch))
  >>> for i in range(20):
  ...     loss = model.fit_sequences(data)
  >>> prediction = model.predict_from_sequences(train_smiles, 5)
  
  References
  ----------
  .. [1] Sutskever et al., "Sequence to Sequence Learning with Neural Networks"
class_docstr_tok: ["Implements","sequence","to","sequence","translation","models",".","The","model","is","based","on","the","description","in","Sutskever","et","al.",",","``","Sequence","to","Sequence","Learning","with","Neural","Networks","''","(","https",":","\/\/arxiv.org\/abs\/1409.3215",")",",","although","this","implementation","uses","GRUs","instead","of","LSTMs",".","The","goal","is","to","take","sequences","of","tokens","as","input",",","and","translate","each","one","into","a","different","output","sequence",".","The","input","and","output","sequences","can","both","be","of","variable","length",",","and","an","output","sequence","need","not","have","the","same","length","as","the","input","sequence","it","was","generated","from",".","For","example",",","these","models","were","originally","developed","for","use","in","natural","language","processing",".","In","that","context",",","the","input","might","be","a","sequence","of","English","words",",","and","the","output","might","be","a","sequence","of","French","words",".","The","goal","would","be","to","train","the","model","to","translate","sentences","from","English","to","French",".","The","model","consists","of","two","parts","called","the","``","encoder","''","and","``","decoder","''",".","Each","one","consists","of","a","stack","of","recurrent","layers",".","The","job","of","the","encoder","is","to","transform","the","input","sequence","into","a","single",",","fixed","length","vector","called","the","''","embedding","''",".","That","vector","contains","all","relevant","information","from","the","input","sequence",".","The","decoder","then","transforms","the","embedding","vector","into","the","output","sequence",".","These","models","can","be","used","for","various","purposes",".","First","and","most","obviously",",","they","can","be","used","for","sequence","to","sequence","translation",".","In","any","case","where","you","have","sequences","of","tokens",",","and","you","want","to","translate","each","one","into","a","different","sequence",",","a","SeqToSeq","model","can","be","trained","to","perform","the","translation",".","Another","possible","use","case","is","transforming","variable","length","sequences","into","fixed","length","vectors",".","Many","types","of","models","require","their","inputs","to","have","a","fixed","shape",",","which","makes","it","difficult","to","use","them","with","variable","sized","inputs","(","for","example",",","when","the","input","is","a","molecule",",","and","different","molecules","have","different","numbers","of","atoms",")",".","In","that","case",",","you","can","train","a","SeqToSeq","model","as","an","autoencoder",",","so","that","it","tries","to","make","the","output","sequence","identical","to","the","input","one",".","That","forces","the","embedding","vector","to","contain","all","information","from","the","original","sequence",".","You","can","then","use","the","encoder","for","transforming","sequences","into","fixed","length","embedding","vectors",",","suitable","to","use","as","inputs","to","other","types","of","models",".","Another","use","case","is","to","train","the","decoder","for","use","as","a","generative","model",".","Here","again","you","begin","by","training","the","SeqToSeq","model","as","an","autoencoder",".","Once","training","is","complete",",","you","can","supply","arbitrary","embedding","vectors",",","and","transform","each","one","into","an","output","sequence",".","When","used","in","this","way",",","you","typically","train","it","as","a","variational","autoencoder",".","This","adds","random","noise","to","the","encoder",",","and","also","adds","a","constraint","term","to","the","loss","that","forces","the","embedding","vector","to","have","a","unit","Gaussian","distribution",".","You","can","then","pick","random","vectors","from","a","Gaussian","distribution",",","and","the","output","sequences","should","follow","the","same","distribution","as","the","training","data",".","When","training","as","a","variational","autoencoder",",","it","is","best","to","use","KL","cost","annealing",",","as","described","in","https",":","\/\/arxiv.org\/abs\/1511.06349",".","The","constraint","term","in","the","loss","is","initially","set","to","0",",","so","the","optimizer","just","tries","to","minimize","the","reconstruction","loss",".","Once","it","has","made","reasonable","progress","toward","that",",","the","constraint","term","can","be","gradually","turned","back","on",".","The","range","of","steps","over","which","this","happens","is","configurable",".","In","this","class",",","we","establish","a","sequential","model","for","the","Sequence","to","Sequence","(","DTNN",")","[","1","]","_",".","Examples","--","--","--","--",">",">",">","import","torch",">",">",">","from","deepchem.models.torch_models.seqtoseq","import","SeqToSeqModel",">",">",">","data","=","[","...","(","``","Cc1cccc","(","N2CCN","(","C","(","=O",")","C34CC5CC","(","CC","(","C5",")","C3",")","C4",")","CC2",")","c1C","''",",","...","``","Cc1cccc","(","N2CCN","(","C","(","=O",")","C34CC5CC","(","CC","(","C5",")","C3",")","C4",")","CC2",")","c1C","''",")",",","...","(","``","Cn1ccnc1SCC","(","=O",")","Nc1ccc","(","Oc2ccccc2",")","cc1","''",",","...","``","Cn1ccnc1SCC","(","=O",")","Nc1ccc","(","Oc2ccccc2",")","cc1","''",")",",","...","(","``","COc1cc2c","(","cc1NC","(","=O",")","CN1C","(","=O",")","NC3","(","CCc4ccccc43",")","C1=O",")","oc1ccccc12","''",",","...","``","COc1cc2c","(","cc1NC","(","=O",")","CN1C","(","=O",")","NC3","(","CCc4ccccc43",")","C1=O",")","oc1ccccc12","''",")",",","...","(","``","O=C1\/C","(","=C\/NC2CCS","(","=O",")","(","=O",")","C2",")","c2ccccc2C","(","=O",")","N1c1ccccc1","''",",","...","``","O=C1\/C","(","=C\/NC2CCS","(","=O",")","(","=O",")","C2",")","c2ccccc2C","(","=O",")","N1c1ccccc1","''",")",",","...","(","``","NC","(","=O",")","NC","(","Cc1ccccc1",")","C","(","=O",")","O","''",",","...","``","NC","(","=O",")","NC","(","Cc1ccccc1",")","C","(","=O",")","O","''",")","]",">",">",">","train_smiles","=","[","s","[","0","]","for","s","in","data","]",">",">",">","tokens","=","set","(",")",">",">",">","for","s","in","train_smiles",":","...","tokens","=","tokens.union","(","set","(","c","for","c","in","s",")",")",">",">",">","tokens","=","sorted","(","list","(","tokens",")",")",">",">",">","from","deepchem.models.optimizers","import","Adam",",","ExponentialDecay",">",">",">","max_length","=","max","(","len","(","s",")","for","s","in","train_smiles",")",">",">",">","batch_size","=","100",">",">",">","batches_per_epoch","=","len","(","train_smiles",")","\/","batch_size",">",">",">","model","=","SeqToSeqModel","(","...","tokens",",","...","tokens",",","...","max_length",",","...","encoder_layers=2",",","...","decoder_layers=2",",","...","embedding_dimension=256",",","...","model_dir=","''","fingerprint","''",",","...","batch_size=batch_size",",","...","learning_rate=ExponentialDecay","(","0.001",",","0.9",",","batches_per_epoch",")",")",">",">",">","for","i","in","range","(","20",")",":","...","loss","=","model.fit_sequences","(","data",")",">",">",">","prediction","=","model.predict_from_sequences","(","train_smiles",",","5",")","References","--","--","--","--","--","..","[","1","]","Sutskever","et","al.",",","``","Sequence","to","Sequence","Learning","with","Neural","Networks","''"]
docstr: Train this model on a set of sequences
  
  Parameters
  ----------
  sequences: List[str]
      Training samples to fit to. Each sample should be represented
      as a tuple of the form (input_sequence, output_sequence).
  max_checkpoints_to_keep: int
      Maximum number of checkpoints to keep. Older checkpoints are
      discarded.
  checkpoint_interval: int
      Frequency at which to write checkpoints, measured in training steps.
  restore: bool
      if True, restore the model from the most recent checkpoint and
      continue training from there. If False, retrain the model from
      scratch.
docstr_tok: ["Train","this","model","on","a","set","of","sequences","Parameters","--","--","--","--","--","sequences",":","List","[","str","]","Training","samples","to","fit","to",".","Each","sample","should","be","represented","as","a","tuple","of","the","form","(","input_sequence",",","output_sequence",")",".","max_checkpoints_to_keep",":","int","Maximum","number","of","checkpoints","to","keep",".","Older","checkpoints","are","discarded",".","checkpoint_interval",":","int","Frequency","at","which","to","write","checkpoints",",","measured","in","training","steps",".","restore",":","bool","if","True",",","restore","the","model","from","the","most","recent","checkpoint","and","continue","training","from","there",".","If","False",",","retrain","the","model","from","scratch","."]
code: def fit_sequences(
      self,
      sequences: List[str],
      max_checkpoints_to_keep: int = 5,
      checkpoint_interval: int = 1000,
      restore: bool = False,
  ):
      """Train this model on a set of sequences
  
      Parameters
      ----------
      sequences: List[str]
          Training samples to fit to. Each sample should be represented
          as a tuple of the form (input_sequence, output_sequence).
      max_checkpoints_to_keep: int
          Maximum number of checkpoints to keep. Older checkpoints are
          discarded.
      checkpoint_interval: int
          Frequency at which to write checkpoints, measured in training steps.
      restore: bool
          if True, restore the model from the most recent checkpoint and
          continue training from there. If False, retrain the model from
          scratch.
  
      """
      loss = self.fit_generator(
          self._generate_batches(sequences),
          max_checkpoints_to_keep=max_checkpoints_to_keep,
          checkpoint_interval=checkpoint_interval,
          restore=restore,
      )
      return loss
  
imports: from heapq import heappush, heappushpop
  from typing import List
  import numpy
  import torch.nn
  import torch
  from deepchem.models.torch_models.layers import EncoderRNN, DecoderRNN, VariationalRandomizer
  from deepchem.models.torch_models import TorchModel
  from deepchem.utils.batch_utils import batch_elements, create_input_array, create_output_array
label_desc: Use image node_id 3 for calling the SeqToSeqModel obj's underlying member method code with example usage: obj.fit_sequences(sequences, max_checkpoints_to_keep, checkpoint_interval, restore) and returns: loss