class_docstr: An abstract class represening HadoopDataset which loads data from hdfs.
  All subclasses should overwrite :code:`__iter__`.
  
  Examples:
      .. code-block:: python
  
          from pgl.utils.data import HadoopDataset
  
          class MyHadoopDataset(HadoopDataset):
              def __init__(self, data_path, hadoop_bin, fs_name, fs_ugi):
                  super(MyHadoopDataset, self).__init__(hadoop_bin, fs_name, fs_ugi)
                  self.data_path = data_path
  
              def __iter__(self):
                  for line in self._line_data_generator():
                      yield line    
  
              def _line_data_generator(self):
                  paths = self.hadoop_util.ls(self.data_path)
                  paths = sorted(paths)
                  for idx, filename in enumerate(paths):
                      if idx % self._worker_info.num_workers != self._worker_info.fid:
                          continue
                      with self.hadoop_util.open(filename) as f:
                          for line in f:
                              yield line
code: def __init__(self, hadoop_bin, fs_name, fs_ugi):
      self.hadoop_util = HadoopUtil(
          hadoop_bin=hadoop_bin, fs_name=fs_name, fs_ugi=fs_ugi
      )
  
imports: import os
  import sys
  import numpy
  import json
  import io
  from subprocess import Popen, PIPE
label_desc: Use image node_id 1 to create a new HadoopDataset object from inherited base classes: StreamDataset with example: obj = HadoopDataset(hadoop_bin, fs_name, fs_ugi)