class_docstr: A class implementing the chatbot problem for the OpenSubtitles dataset.
  
  http://opus.nlpl.eu/OpenSubtitles-v2018.php
class_docstr_tok: ["A","class","implementing","the","chatbot","problem","for","the","OpenSubtitles","dataset",".","http",":","\/\/opus.nlpl.eu\/OpenSubtitles-v2018.php"]
docstr: Main function where the preprocessing of the data starts.
  
  Args:
    train_mode: string, whether we are in train, dev or test mode
docstr_tok: ["Main","function","where","the","preprocessing","of","the","data","starts",".","Args",":","train_mode",":","string",",","whether","we","are","in","train",",","dev","or","test","mode"]
code: def preprocess_data(self, train_mode):
      """Main function where the preprocessing of the data starts.
  
      Args:
        train_mode: string, whether we are in train, dev or test mode
      """
  
      year = (
          ""
          if self.dataset_version == 2009
          else str(self.dataset_version)
      )
      # Set the raw data directory and data.
      self.raw_data_dir = os.path.join(
          "/".join(self._data_dir.split("/")[:-1]),
          "raw_data_" + str(self.dataset_version),
      )
      self.raw_data = os.path.join(
          self._raw_data_dir, "OpenSubtitles" + year
      )
      self.zipped_data = os.path.join(self._raw_data_dir, "en.tar.gz")
  
      # Create the download url.
      self.url = (
          "http://opus.nlpl.eu/download.php?f=OpenSubtitles"
          + str(year)
          + "/en.tar.gz"
      )
  
      # Check at which part of the pipeline are we at.
      self.data_pipeline_status(train_mode)
  
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import collections
  import os
  import re
  import zipfile
  from tensor2tensor.data_generators import dialog_abstract
  from tensor2tensor.data_generators import text_encoder
  from tensor2tensor.utils import registry
label_desc: Use image node_id 4 for calling the DialogOpensubtitles64k2009 obj's underlying member method code with example usage: obj.preprocess_data(train_mode) without return types