prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  Radiology DataLoader takes up a *lot* memory during certain conditions
  **Describe the bug**
  During sanity checking of subjects, the queue construction seems to take up a lot of memory.
  
  **To Reproduce**
  Steps to reproduce the behavior:
  1. Have a ridiculous number of subjects on a small machine (e.g., 10k on a machine with 16G RAM)
  2. Start training on rad mode
  4. See error sometime during/after queue construction:
  ```bash
  ## last message
  Constructing queue for train data: 100%|██████████| 8681/8681 [07:57<00:00, 18.19it/s]
  ## failure with message related to exceeded RAM usage
  ```
  
  **Expected behavior**
  There should not be any failure at this stage.
  
  **Screenshots**
  N.A>
  
  **GaNDLF Version**
  <!-- Put the output of the following command:
  python -c 'import GANDLF as g;print(g.__version__)'
  -->
  0.0.16-dev
  
  **Desktop (please complete the following information):**
  CentOS 7
  
  **Additional context**
  N.A.
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `setup.py`
  Content:
  ```
  1 #!/usr/bin/env python
  2 
  3 """The setup script."""
  4 
  5 
  6 import os
  7 from setuptools import setup, find_packages
  8 from setuptools.command.install import install
  9 from setuptools.command.develop import develop
  10 from setuptools.command.egg_info import egg_info
  11 
  12 with open("README.md") as readme_file:
  13     readme = readme_file.read()
  14 
  15 
  16 def git_submodule_update():
  17     ## submodule update
  18     os.system("git submodule update --init --recursive")
  19 
  20 
  21 class CustomInstallCommand(install):
  22     def run(self):
  23         install.run(self)
  24         git_submodule_update()
  25 
  26 
  27 class CustomDevelopCommand(develop):
  28     def run(self):
  29         develop.run(self)
  30         git_submodule_update()
  31 
  32 
  33 class CustomEggInfoCommand(egg_info):
  34     def run(self):
  35         egg_info.run(self)
  36         git_submodule_update()
  37 
  38 
  39 # read version.py
  40 import sys, re
  41 
  42 try:
  43     filepath = "GANDLF/version.py"
  44     version_file = open(filepath)
  45     (__version__,) = re.findall('__version__ = "(.*)"', version_file.read())
  46 
  47 except Exception as error:
  48     __version__ = "0.0.1"
  49     sys.stderr.write("Warning: Could not open '%s' due %s\n" % (filepath, error))
  50 
  51 requirements = [
  52     "black",
  53     "numpy==1.22.0",
  54     "scipy",
  55     "SimpleITK!=2.0.*",
  56     "torchvision",
  57     "tqdm",
  58     "torchio==0.18.75",
  59     "pandas",
  60     "scikit-learn>=0.23.2",
  61     "scikit-image>=0.19.1",
  62     'pickle5>=0.0.11; python_version < "3.8.0"',
  63     "setuptools",
  64     "seaborn",
  65     "pyyaml",
  66     "tiffslide",
  67     "matplotlib",
  68     "requests>=2.25.0",
  69     "pytest",
  70     "coverage",
  71     "pytest-cov",
  72     "psutil",
  73     "medcam",
  74     "opencv-python",
  75     "torchmetrics==0.5.1",  # newer versions have changed api for f1 invocation
  76     "OpenPatchMiner==0.1.8",
  77     "zarr==2.10.3",
  78     "pydicom",
  79     "onnx",
  80     "torchinfo==1.7.0",
  81     "segmentation-models-pytorch==0.3.0",
  82     "ACSConv==0.1.1",
  83 ]
  84 
  85 # pytorch doesn't have LTS support on OSX - https://github.com/mlcommons/GaNDLF/issues/389
  86 if sys.platform == "darwin":
  87     requirements.append("torch==1.11.0")
  88 else:
  89     requirements.append("torch==1.11.0")
  90 
  91 setup(
  92     name="GANDLF",
  93     version=__version__,
  94     author="MLCommons",
  95     author_email="gandlf@mlcommons.org",
  96     python_requires=">=3.7",
  97     packages=find_packages(),
  98     cmdclass={  # this ensures git_submodule_update is called during install
  99         "install": CustomInstallCommand,
  100         "develop": CustomDevelopCommand,
  101         "egg_info": CustomEggInfoCommand,
  102     },
  103     scripts=[
  104         "gandlf_run",
  105         "gandlf_constructCSV",
  106         "gandlf_collectStats",
  107         "gandlf_patchMiner",
  108         "gandlf_preprocess",
  109         "gandlf_anonymizer",
  110         "gandlf_verifyInstall",
  111         "gandlf_configGenerator",
  112     ],
  113     classifiers=[
  114         "Development Status :: 3 - Alpha",
  115         "Intended Audience :: Science/Research",
  116         "License :: OSI Approved :: Apache Software License",
  117         "Natural Language :: English",
  118         "Operating System :: OS Independent",
  119         "Programming Language :: Python :: 3.7",
  120         "Programming Language :: Python :: 3.8",
  121         "Programming Language :: Python :: 3.9",
  122         "Programming Language :: Python :: 3.10",
  123         "Topic :: Scientific/Engineering :: Medical Science Apps",
  124     ],
  125     description=(
  126         "PyTorch-based framework that handles segmentation/regression/classification using various DL architectures for medical imaging."
  127     ),
  128     install_requires=requirements,
  129     license="Apache-2.0",
  130     long_description=readme,
  131     long_description_content_type="text/markdown",
  132     include_package_data=True,
  133     keywords="semantic, segmentation, regression, classification, data-augmentation, medical-imaging, clinical-workflows, deep-learning, pytorch",
  134     zip_safe=False,
  135 )
  136
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/setup.py b/setup.py
  --- a/setup.py
  +++ b/setup.py
  @@ -53,6 +53,7 @@
       "numpy==1.22.0",
       "scipy",
       "SimpleITK!=2.0.*",
  +    "SimpleITK!=2.2.1",  # https://github.com/mlcommons/GaNDLF/issues/536
       "torchvision",
       "tqdm",
       "torchio==0.18.75",
  
verification_info: {"golden_diff": "diff --git a/setup.py b/setup.py\n--- a/setup.py\n+++ b/setup.py\n@@ -53,6 +53,7 @@\n     \"numpy==1.22.0\",\n     \"scipy\",\n     \"SimpleITK!=2.0.*\",\n+    \"SimpleITK!=2.2.1\",  # https://github.com/mlcommons/GaNDLF/issues/536\n     \"torchvision\",\n     \"tqdm\",\n     \"torchio==0.18.75\",\n", "issue": "Radiology DataLoader takes up a *lot* memory during certain conditions\n**Describe the bug**\r\nDuring sanity checking of subjects, the queue construction seems to take up a lot of memory.\r\n\r\n**To Reproduce**\r\nSteps to reproduce the behavior:\r\n1. Have a ridiculous number of subjects on a small machine (e.g., 10k on a machine with 16G RAM)\r\n2. Start training on rad mode\r\n4. See error sometime during/after queue construction:\r\n```bash\r\n## last message\r\nConstructing queue for train data: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 8681/8681 [07:57<00:00, 18.19it/s]\r\n## failure with message related to exceeded RAM usage\r\n```\r\n\r\n**Expected behavior**\r\nThere should not be any failure at this stage.\r\n\r\n**Screenshots**\r\nN.A>\r\n\r\n**GaNDLF Version**\r\n<!-- Put the output of the following command:\r\npython -c 'import GANDLF as g;print(g.__version__)'\r\n-->\r\n0.0.16-dev\r\n\r\n**Desktop (please complete the following information):**\r\nCentOS 7\r\n\r\n**Additional context**\r\nN.A.\n", "before_files": [{"content": "#!/usr/bin/env python\n\n\"\"\"The setup script.\"\"\"\n\n\nimport os\nfrom setuptools import setup, find_packages\nfrom setuptools.command.install import install\nfrom setuptools.command.develop import develop\nfrom setuptools.command.egg_info import egg_info\n\nwith open(\"README.md\") as readme_file:\n    readme = readme_file.read()\n\n\ndef git_submodule_update():\n    ## submodule update\n    os.system(\"git submodule update --init --recursive\")\n\n\nclass CustomInstallCommand(install):\n    def run(self):\n        install.run(self)\n        git_submodule_update()\n\n\nclass CustomDevelopCommand(develop):\n    def run(self):\n        develop.run(self)\n        git_submodule_update()\n\n\nclass CustomEggInfoCommand(egg_info):\n    def run(self):\n        egg_info.run(self)\n        git_submodule_update()\n\n\n# read version.py\nimport sys, re\n\ntry:\n    filepath = \"GANDLF/version.py\"\n    version_file = open(filepath)\n    (__version__,) = re.findall('__version__ = \"(.*)\"', version_file.read())\n\nexcept Exception as error:\n    __version__ = \"0.0.1\"\n    sys.stderr.write(\"Warning: Could not open '%s' due %s\\n\" % (filepath, error))\n\nrequirements = [\n    \"black\",\n    \"numpy==1.22.0\",\n    \"scipy\",\n    \"SimpleITK!=2.0.*\",\n    \"torchvision\",\n    \"tqdm\",\n    \"torchio==0.18.75\",\n    \"pandas\",\n    \"scikit-learn>=0.23.2\",\n    \"scikit-image>=0.19.1\",\n    'pickle5>=0.0.11; python_version < \"3.8.0\"',\n    \"setuptools\",\n    \"seaborn\",\n    \"pyyaml\",\n    \"tiffslide\",\n    \"matplotlib\",\n    \"requests>=2.25.0\",\n    \"pytest\",\n    \"coverage\",\n    \"pytest-cov\",\n    \"psutil\",\n    \"medcam\",\n    \"opencv-python\",\n    \"torchmetrics==0.5.1\",  # newer versions have changed api for f1 invocation\n    \"OpenPatchMiner==0.1.8\",\n    \"zarr==2.10.3\",\n    \"pydicom\",\n    \"onnx\",\n    \"torchinfo==1.7.0\",\n    \"segmentation-models-pytorch==0.3.0\",\n    \"ACSConv==0.1.1\",\n]\n\n# pytorch doesn't have LTS support on OSX - https://github.com/mlcommons/GaNDLF/issues/389\nif sys.platform == \"darwin\":\n    requirements.append(\"torch==1.11.0\")\nelse:\n    requirements.append(\"torch==1.11.0\")\n\nsetup(\n    name=\"GANDLF\",\n    version=__version__,\n    author=\"MLCommons\",\n    author_email=\"gandlf@mlcommons.org\",\n    python_requires=\">=3.7\",\n    packages=find_packages(),\n    cmdclass={  # this ensures git_submodule_update is called during install\n        \"install\": CustomInstallCommand,\n        \"develop\": CustomDevelopCommand,\n        \"egg_info\": CustomEggInfoCommand,\n    },\n    scripts=[\n        \"gandlf_run\",\n        \"gandlf_constructCSV\",\n        \"gandlf_collectStats\",\n        \"gandlf_patchMiner\",\n        \"gandlf_preprocess\",\n        \"gandlf_anonymizer\",\n        \"gandlf_verifyInstall\",\n        \"gandlf_configGenerator\",\n    ],\n    classifiers=[\n        \"Development Status :: 3 - Alpha\",\n        \"Intended Audience :: Science/Research\",\n        \"License :: OSI Approved :: Apache Software License\",\n        \"Natural Language :: English\",\n        \"Operating System :: OS Independent\",\n        \"Programming Language :: Python :: 3.7\",\n        \"Programming Language :: Python :: 3.8\",\n        \"Programming Language :: Python :: 3.9\",\n        \"Programming Language :: Python :: 3.10\",\n        \"Topic :: Scientific/Engineering :: Medical Science Apps\",\n    ],\n    description=(\n        \"PyTorch-based framework that handles segmentation/regression/classification using various DL architectures for medical imaging.\"\n    ),\n    install_requires=requirements,\n    license=\"Apache-2.0\",\n    long_description=readme,\n    long_description_content_type=\"text/markdown\",\n    include_package_data=True,\n    keywords=\"semantic, segmentation, regression, classification, data-augmentation, medical-imaging, clinical-workflows, deep-learning, pytorch\",\n    zip_safe=False,\n)\n", "path": "setup.py"}], "after_files": [{"content": "#!/usr/bin/env python\n\n\"\"\"The setup script.\"\"\"\n\n\nimport os\nfrom setuptools import setup, find_packages\nfrom setuptools.command.install import install\nfrom setuptools.command.develop import develop\nfrom setuptools.command.egg_info import egg_info\n\nwith open(\"README.md\") as readme_file:\n    readme = readme_file.read()\n\n\ndef git_submodule_update():\n    ## submodule update\n    os.system(\"git submodule update --init --recursive\")\n\n\nclass CustomInstallCommand(install):\n    def run(self):\n        install.run(self)\n        git_submodule_update()\n\n\nclass CustomDevelopCommand(develop):\n    def run(self):\n        develop.run(self)\n        git_submodule_update()\n\n\nclass CustomEggInfoCommand(egg_info):\n    def run(self):\n        egg_info.run(self)\n        git_submodule_update()\n\n\n# read version.py\nimport sys, re\n\ntry:\n    filepath = \"GANDLF/version.py\"\n    version_file = open(filepath)\n    (__version__,) = re.findall('__version__ = \"(.*)\"', version_file.read())\n\nexcept Exception as error:\n    __version__ = \"0.0.1\"\n    sys.stderr.write(\"Warning: Could not open '%s' due %s\\n\" % (filepath, error))\n\nrequirements = [\n    \"black\",\n    \"numpy==1.22.0\",\n    \"scipy\",\n    \"SimpleITK!=2.0.*\",\n    \"SimpleITK!=2.2.1\",  # https://github.com/mlcommons/GaNDLF/issues/536\n    \"torchvision\",\n    \"tqdm\",\n    \"torchio==0.18.75\",\n    \"pandas\",\n    \"scikit-learn>=0.23.2\",\n    \"scikit-image>=0.19.1\",\n    'pickle5>=0.0.11; python_version < \"3.8.0\"',\n    \"setuptools\",\n    \"seaborn\",\n    \"pyyaml\",\n    \"tiffslide\",\n    \"matplotlib\",\n    \"requests>=2.25.0\",\n    \"pytest\",\n    \"coverage\",\n    \"pytest-cov\",\n    \"psutil\",\n    \"medcam\",\n    \"opencv-python\",\n    \"torchmetrics==0.5.1\",  # newer versions have changed api for f1 invocation\n    \"OpenPatchMiner==0.1.8\",\n    \"zarr==2.10.3\",\n    \"pydicom\",\n    \"onnx\",\n    \"torchinfo==1.7.0\",\n    \"segmentation-models-pytorch==0.3.0\",\n    \"ACSConv==0.1.1\",\n]\n\n# pytorch doesn't have LTS support on OSX - https://github.com/mlcommons/GaNDLF/issues/389\nif sys.platform == \"darwin\":\n    requirements.append(\"torch==1.11.0\")\nelse:\n    requirements.append(\"torch==1.11.0\")\n\nsetup(\n    name=\"GANDLF\",\n    version=__version__,\n    author=\"MLCommons\",\n    author_email=\"gandlf@mlcommons.org\",\n    python_requires=\">=3.7\",\n    packages=find_packages(),\n    cmdclass={  # this ensures git_submodule_update is called during install\n        \"install\": CustomInstallCommand,\n        \"develop\": CustomDevelopCommand,\n        \"egg_info\": CustomEggInfoCommand,\n    },\n    scripts=[\n        \"gandlf_run\",\n        \"gandlf_constructCSV\",\n        \"gandlf_collectStats\",\n        \"gandlf_patchMiner\",\n        \"gandlf_preprocess\",\n        \"gandlf_anonymizer\",\n        \"gandlf_verifyInstall\",\n        \"gandlf_configGenerator\",\n    ],\n    classifiers=[\n        \"Development Status :: 3 - Alpha\",\n        \"Intended Audience :: Science/Research\",\n        \"License :: OSI Approved :: Apache Software License\",\n        \"Natural Language :: English\",\n        \"Operating System :: OS Independent\",\n        \"Programming Language :: Python :: 3.7\",\n        \"Programming Language :: Python :: 3.8\",\n        \"Programming Language :: Python :: 3.9\",\n        \"Programming Language :: Python :: 3.10\",\n        \"Topic :: Scientific/Engineering :: Medical Science Apps\",\n    ],\n    description=(\n        \"PyTorch-based framework that handles segmentation/regression/classification using various DL architectures for medical imaging.\"\n    ),\n    install_requires=requirements,\n    license=\"Apache-2.0\",\n    long_description=readme,\n    long_description_content_type=\"text/markdown\",\n    include_package_data=True,\n    keywords=\"semantic, segmentation, regression, classification, data-augmentation, medical-imaging, clinical-workflows, deep-learning, pytorch\",\n    zip_safe=False,\n)\n", "path": "setup.py"}]}