title: How to create a custom data loader in Pytorch?
body: <p>I have a file containing paths to images I would like to load into Pytorch, while utilizing the built-in dataloader features (multiprocess loading pipeline, data augmentations, and so on).</p>
  <pre><code>def create_links():
      data_dir = &quot;/myfolder&quot;
  
      full_path_list = []
      assert os.path.isdir(data_dir)
      for _, _, filenames in os.walk(data_dir):
          for filename in filenames:
              full_path_list.append(os.path.join(data_dir, filename))
  
      with open(config.data.links_file, 'w+') as links_file:
          for full_path in full_path_list:
              links_file.write(f&quot;{full_path}\n&quot;)
  
  
  def read_links_file_to_list():
      config = ConfigProvider.config()
      links_file_path = config.data.links_file
      if not os.path.isfile(links_file_path):
          raise RuntimeError(&quot;did you forget to create a file with links to images? Try using 'create_links()'&quot;)
      with open(links_file_path, 'r') as links_file:
          return links_file.readlines()
  </code></pre>
  <p>So I have a list of files (or a generator, or whatever works), <code>file_list = read_links_file_to_list()</code>.</p>
  <p>How can I build a Pytorch dataloader around it, and how would I use it?</p>