class_docstr: Open Retrieval Question Answering evaluation using Google NQ dataset.
class_docstr_tok: ["Open","Retrieval","Question","Answering","evaluation","using","Google","NQ","dataset","."]
code: def process_samples_from_single_path(filename):
      print_rank_0(" > Processing {} ...".format(filename))
      samples = []
      total = 0
  
      with open(filename, "r") as ifile:
          reader = csv.reader(ifile, delimiter="\t")
          for row in reader:
              question = row[0]
              answers = eval(row[1])
  
              sample = {"question": question, "answers": answers}
              total += 1
              samples.append(sample)
  
              if total % 1000 == 0:
                  print_rank_0(
                      "  > processed {} so far ...".format(total)
                  )
  
      print_rank_0(" >> processed {} samples.".format(len(samples)))
      return samples
  
imports: from abc import ABC
  import csv
  from collections import OrderedDict
  import numpy
  import torch
  from torch.utils.data import DataLoader
  from torch.utils.data import Dataset, BatchSampler
  from megatron import print_rank_0, get_args, get_tokenizer
  from megatron.data.biencoder_dataset_utils import make_attention_mask
label_desc: Use image node_id 4 for calling the NQDataset obj's underlying member method code with example usage: obj.process_samples_from_single_path(filename) and returns: samples