class_docstr: A `SparkDatasetConverter` object holds one materialized spark dataframe and
  can be used to make one or more tensorflow datasets or torch dataloaders.
  The `SparkDatasetConverter` object is picklable and can be used in remote
  processes.
  See `make_spark_converter`
class_docstr_tok: ["A","`","SparkDatasetConverter","`","object","holds","one","materialized","spark","dataframe","and","can","be","used","to","make","one","or","more","tensorflow","datasets","or","torch","dataloaders",".","The","`","SparkDatasetConverter","`","object","is","picklable","and","can","be","used","in","remote","processes",".","See","`","make_spark_converter","`"]
docstr: Make a PyTorch DataLoader.
  
  This method will do the following two steps:
    1) Open a petastorm reader on the materialized dataset dir.
    2) Create a PyTorch DataLoader based on the reader created in (1)
  
  :param batch_size: The number of items to return per batch. Default ``None``.
      If ``None``, current implementation will set batch size to be 32, in future,
      ``None`` value will denotes auto tuned best value for batch size.
  :param num_epochs: An epoch is a single pass over all rows in the
      dataset. Setting ``num_epochs`` to ``None`` will result in an
      infinite number of epochs.
  :param workers_count: An int for the number of workers to use in the
      reader pool. This only is used for the thread or process pool.
      Defaults value ``None``, which means using the default value from
      `petastorm.make_batch_reader()`. We can autotune it in the future.
  :param shuffling_queue_capacity: Queue capacity is passed to the underlying
      :class:`tf.RandomShuffleQueue` instance. If set to 0, no shuffling will be done.
  :param data_loader_fn: A function (or class) that generates a
      `torch.utils.data.DataLoader` object. The default value of ``None`` uses
      `petastorm.pytorch.DataLoader`.
  :param petastorm_reader_kwargs: arguments for `petastorm.make_batch_reader()`,
      exclude these arguments: ``dataset_url``, ``num_epochs``, ``workers_count``.
  
  :return: a context manager for a `torch.utils.data.DataLoader` object.
           when exit the returned context manager, the reader
           will be closed.
code: def make_torch_dataloader(
      self,
      batch_size=32,
      num_epochs=None,
      workers_count=None,
      shuffling_queue_capacity=0,
      data_loader_fn=None,
      **petastorm_reader_kwargs,
  ):
      """Make a PyTorch DataLoader.
  
      This method will do the following two steps:
        1) Open a petastorm reader on the materialized dataset dir.
        2) Create a PyTorch DataLoader based on the reader created in (1)
  
      :param batch_size: The number of items to return per batch. Default ``None``.
          If ``None``, current implementation will set batch size to be 32, in future,
          ``None`` value will denotes auto tuned best value for batch size.
      :param num_epochs: An epoch is a single pass over all rows in the
          dataset. Setting ``num_epochs`` to ``None`` will result in an
          infinite number of epochs.
      :param workers_count: An int for the number of workers to use in the
          reader pool. This only is used for the thread or process pool.
          Defaults value ``None``, which means using the default value from
          `petastorm.make_batch_reader()`. We can autotune it in the future.
      :param shuffling_queue_capacity: Queue capacity is passed to the underlying
          :class:`tf.RandomShuffleQueue` instance. If set to 0, no shuffling will be done.
      :param data_loader_fn: A function (or class) that generates a
          `torch.utils.data.DataLoader` object. The default value of ``None`` uses
          `petastorm.pytorch.DataLoader`.
      :param petastorm_reader_kwargs: arguments for `petastorm.make_batch_reader()`,
          exclude these arguments: ``dataset_url``, ``num_epochs``, ``workers_count``.
  
      :return: a context manager for a `torch.utils.data.DataLoader` object.
               when exit the returned context manager, the reader
               will be closed.
      """
      self._check_and_set_overriden_petastorm_args(
          petastorm_reader_kwargs,
          num_epochs=num_epochs,
          workers_count=workers_count,
      )
      return TorchDatasetContextManager(
          self.file_urls,
          batch_size=batch_size,
          petastorm_reader_kwargs=petastorm_reader_kwargs,
          shuffling_queue_capacity=shuffling_queue_capacity,
          data_loader_fn=data_loader_fn,
      )
  
imports: import atexit
  import datetime
  import logging
  import os
  import shutil
  import threading
  import time
  import uuid
  from distutils.version import LooseVersion
  from multiprocessing.pool import ThreadPool
  from typing import List, Any
  import pyspark
  from pyarrow import LocalFileSystem
  from pyspark.sql.session import SparkSession
  from pyspark.sql.types import ArrayType, DoubleType, FloatType
  from six.moves.urllib.parse import urlparse
  from petastorm import make_batch_reader
  from petastorm.fs_utils import FilesystemResolver, get_filesystem_and_path_or_paths, normalize_dir_url
  from fsspec.core import strip_protocol
label_desc: Use image node_id 5 for calling the SparkDatasetConverter obj's underlying member method code with example usage: obj.make_torch_dataloader(batch_size, num_epochs, workers_count, shuffling_queue_capacity, data_loader_fn) and returns: TorchDatasetContextManager