class_docstr: Base information retrieval dataset on which other datasets are built.
  
  Args:
      tokenizer: tokenizer
      max_query_length: maximum length of query in tokens
      max_passage_length: maximum length of passage in tokens
class_docstr_tok: ["Base","information","retrieval","dataset","on","which","other","datasets","are","built",".","Args",":","tokenizer",":","tokenizer","max_query_length",":","maximum","length","of","query","in","tokens","max_passage_length",":","maximum","length","of","passage","in","tokens"]
docstr: Function which constructs a valid input to BERT from tokens.
  
  If only one list of tokens (token_ids1) is passed, the input will be
  [CLS] token_ids1 [SEP]
  
  if two lists of tokens are passed, the input will be
  [CLS] token_ids1 [SEP] token_ids2 [SEP]
docstr_tok: ["Function","which","constructs","a","valid","input","to","BERT","from","tokens",".","If","only","one","list","of","tokens","(","token_ids1",")","is","passed",",","the","input","will","be","[","CLS","]","token_ids1","[","SEP","]","if","two","lists","of","tokens","are","passed",",","the","input","will","be","[","CLS","]","token_ids1","[","SEP","]","token_ids2","[","SEP","]"]
code: def construct_input(
      self, token_ids1, max_seq_length, token_ids2=None
  ):
      """
      Function which constructs a valid input to BERT from tokens.
  
      If only one list of tokens (token_ids1) is passed, the input will be
      [CLS] token_ids1 [SEP]
  
      if two lists of tokens are passed, the input will be
      [CLS] token_ids1 [SEP] token_ids2 [SEP]
      """
  
      input_ids = [self.tokenizer.pad_id] * max_seq_length
      bert_input = (
          [self.tokenizer.cls_id] + token_ids1 + [self.tokenizer.sep_id]
      )
      sentence1_length = len(bert_input)
      if token_ids2 is not None:
          bert_input = bert_input + token_ids2 + [self.tokenizer.sep_id]
  
      bert_input = bert_input[:max_seq_length]
  
      num_nonpad_tokens = len(bert_input)
  
      input_ids[:num_nonpad_tokens] = bert_input
      input_ids = np.array(input_ids, dtype=np.longlong)
      input_mask = input_ids != self.tokenizer.pad_id
      input_type_ids = np.ones_like(input_ids)
      input_type_ids[:sentence1_length] = 0
  
      return input_ids, input_mask, input_type_ids
  
imports: import multiprocessing
  import os
  import pickle
  import random
  from typing import Optional
  import numpy
  from torch.utils.data import Dataset
  from nemo.collections.common.tokenizers.tokenizer_spec import TokenizerSpec
label_desc: Use image node_id 6 for calling the BaseInformationRetrievalDataset obj's underlying member method code with example usage: obj.construct_input(token_ids1, max_seq_length, token_ids2) and returns: input_ids,  input_mask,  input_type_ids