prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  The REST API for deleting files does not remove the file's directory and does not update the Media Library list
  **Environment**: 
  
  - CTFd Version/Commit: 3.2.1
  - Operating System: Docker (`python:3.6-slim-buster`)
  - Web Browser and Version: NA
  
  **What happened?**
  
  I am using the REST API for deleting files (e.g. `"DELETE /api/v1/files/41 HTTP/1.1"`) and it seems to work fine. The file is removed. However, two things do go wrong I think (at least to my expectation).
  
  1. The file's directory (which has a hash based name) is not deleted. This means, after a while there will be a lot of empty directories.
  1. The list of files used by the Media Library is not updated (i.e. file is removed from the list) which means the list grows constantly. The result is a list with many non-existing files as they are deleted.
  
  The REST API returns a successful `200` code which seems to match with the expected behaviour.
  
  **What did you expect to happen?**
  
  When a file is deleted using the REST API, I expect the directory (the hash-based name) to be deleted and that the list used by the Media Library is updated accordingly.
  
  **How to reproduce your issue**
  
  1. Upload a file (e.g. via the web interface or REST API). 
  1. Use the REST API to delete this file.
  1. Check the `upload/` folder and the Media Library for the behaviour described above.
  
  **Any associated stack traces or error logs**
  
  None
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `CTFd/utils/uploads/uploaders.py`
  Content:
  ```
  1 import os
  2 import posixpath
  3 import string
  4 from shutil import copyfileobj
  5 
  6 import boto3
  7 from flask import current_app, redirect, send_file
  8 from flask.helpers import safe_join
  9 from werkzeug.utils import secure_filename
  10 
  11 from CTFd.utils import get_app_config
  12 from CTFd.utils.encoding import hexencode
  13 
  14 
  15 class BaseUploader(object):
  16     def __init__(self):
  17         raise NotImplementedError
  18 
  19     def store(self, fileobj, filename):
  20         raise NotImplementedError
  21 
  22     def upload(self, file_obj, filename):
  23         raise NotImplementedError
  24 
  25     def download(self, filename):
  26         raise NotImplementedError
  27 
  28     def delete(self, filename):
  29         raise NotImplementedError
  30 
  31     def sync(self):
  32         raise NotImplementedError
  33 
  34 
  35 class FilesystemUploader(BaseUploader):
  36     def __init__(self, base_path=None):
  37         super(BaseUploader, self).__init__()
  38         self.base_path = base_path or current_app.config.get("UPLOAD_FOLDER")
  39 
  40     def store(self, fileobj, filename):
  41         location = os.path.join(self.base_path, filename)
  42         directory = os.path.dirname(location)
  43 
  44         if not os.path.exists(directory):
  45             os.makedirs(directory)
  46 
  47         with open(location, "wb") as dst:
  48             copyfileobj(fileobj, dst, 16384)
  49 
  50         return filename
  51 
  52     def upload(self, file_obj, filename):
  53         if len(filename) == 0:
  54             raise Exception("Empty filenames cannot be used")
  55 
  56         filename = secure_filename(filename)
  57         md5hash = hexencode(os.urandom(16))
  58         file_path = posixpath.join(md5hash, filename)
  59 
  60         return self.store(file_obj, file_path)
  61 
  62     def download(self, filename):
  63         return send_file(safe_join(self.base_path, filename), as_attachment=True)
  64 
  65     def delete(self, filename):
  66         if os.path.exists(os.path.join(self.base_path, filename)):
  67             os.unlink(os.path.join(self.base_path, filename))
  68             return True
  69         return False
  70 
  71     def sync(self):
  72         pass
  73 
  74 
  75 class S3Uploader(BaseUploader):
  76     def __init__(self):
  77         super(BaseUploader, self).__init__()
  78         self.s3 = self._get_s3_connection()
  79         self.bucket = get_app_config("AWS_S3_BUCKET")
  80 
  81     def _get_s3_connection(self):
  82         access_key = get_app_config("AWS_ACCESS_KEY_ID")
  83         secret_key = get_app_config("AWS_SECRET_ACCESS_KEY")
  84         endpoint = get_app_config("AWS_S3_ENDPOINT_URL")
  85         client = boto3.client(
  86             "s3",
  87             aws_access_key_id=access_key,
  88             aws_secret_access_key=secret_key,
  89             endpoint_url=endpoint,
  90         )
  91         return client
  92 
  93     def _clean_filename(self, c):
  94         if c in string.ascii_letters + string.digits + "-" + "_" + ".":
  95             return True
  96 
  97     def store(self, fileobj, filename):
  98         self.s3.upload_fileobj(fileobj, self.bucket, filename)
  99         return filename
  100 
  101     def upload(self, file_obj, filename):
  102         filename = filter(
  103             self._clean_filename, secure_filename(filename).replace(" ", "_")
  104         )
  105         filename = "".join(filename)
  106         if len(filename) <= 0:
  107             return False
  108 
  109         md5hash = hexencode(os.urandom(16))
  110 
  111         dst = md5hash + "/" + filename
  112         self.s3.upload_fileobj(file_obj, self.bucket, dst)
  113         return dst
  114 
  115     def download(self, filename):
  116         key = filename
  117         filename = filename.split("/").pop()
  118         url = self.s3.generate_presigned_url(
  119             "get_object",
  120             Params={
  121                 "Bucket": self.bucket,
  122                 "Key": key,
  123                 "ResponseContentDisposition": "attachment; filename={}".format(
  124                     filename
  125                 ),
  126             },
  127         )
  128         return redirect(url)
  129 
  130     def delete(self, filename):
  131         self.s3.delete_object(Bucket=self.bucket, Key=filename)
  132         return True
  133 
  134     def sync(self):
  135         local_folder = current_app.config.get("UPLOAD_FOLDER")
  136         # If the bucket is empty then Contents will not be in the response
  137         bucket_list = self.s3.list_objects(Bucket=self.bucket).get("Contents", [])
  138 
  139         for s3_key in bucket_list:
  140             s3_object = s3_key["Key"]
  141             # We don't want to download any directories
  142             if s3_object.endswith("/") is False:
  143                 local_path = os.path.join(local_folder, s3_object)
  144                 directory = os.path.dirname(local_path)
  145                 if not os.path.exists(directory):
  146                     os.makedirs(directory)
  147 
  148                 self.s3.download_file(self.bucket, s3_object, local_path)
  149
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/CTFd/utils/uploads/uploaders.py b/CTFd/utils/uploads/uploaders.py
  --- a/CTFd/utils/uploads/uploaders.py
  +++ b/CTFd/utils/uploads/uploaders.py
  @@ -1,7 +1,8 @@
   import os
   import posixpath
   import string
  -from shutil import copyfileobj
  +from pathlib import PurePath
  +from shutil import copyfileobj, rmtree
   
   import boto3
   from flask import current_app, redirect, send_file
  @@ -64,7 +65,8 @@
   
       def delete(self, filename):
           if os.path.exists(os.path.join(self.base_path, filename)):
  -            os.unlink(os.path.join(self.base_path, filename))
  +            file_path = PurePath(filename).parts[0]
  +            rmtree(os.path.join(self.base_path, file_path))
               return True
           return False
  
verification_info: {"golden_diff": "diff --git a/CTFd/utils/uploads/uploaders.py b/CTFd/utils/uploads/uploaders.py\n--- a/CTFd/utils/uploads/uploaders.py\n+++ b/CTFd/utils/uploads/uploaders.py\n@@ -1,7 +1,8 @@\n import os\n import posixpath\n import string\n-from shutil import copyfileobj\n+from pathlib import PurePath\n+from shutil import copyfileobj, rmtree\n \n import boto3\n from flask import current_app, redirect, send_file\n@@ -64,7 +65,8 @@\n \n     def delete(self, filename):\n         if os.path.exists(os.path.join(self.base_path, filename)):\n-            os.unlink(os.path.join(self.base_path, filename))\n+            file_path = PurePath(filename).parts[0]\n+            rmtree(os.path.join(self.base_path, file_path))\n             return True\n         return False\n", "issue": "The REST API for deleting files does not remove the file's directory and does not update the Media Library list\n**Environment**: \r\n\r\n- CTFd Version/Commit: 3.2.1\r\n- Operating System: Docker (`python:3.6-slim-buster`)\r\n- Web Browser and Version: NA\r\n\r\n**What happened?**\r\n\r\nI am using the REST API for deleting files (e.g. `\"DELETE /api/v1/files/41 HTTP/1.1\"`) and it seems to work fine. The file is removed. However, two things do go wrong I think (at least to my expectation).\r\n\r\n1. The file's directory (which has a hash based name) is not deleted. This means, after a while there will be a lot of empty directories.\r\n1. The list of files used by the Media Library is not updated (i.e. file is removed from the list) which means the list grows constantly. The result is a list with many non-existing files as they are deleted.\r\n\r\nThe REST API returns a successful `200` code which seems to match with the expected behaviour.\r\n\r\n**What did you expect to happen?**\r\n\r\nWhen a file is deleted using the REST API, I expect the directory (the hash-based name) to be deleted and that the list used by the Media Library is updated accordingly.\r\n\r\n**How to reproduce your issue**\r\n\r\n1. Upload a file (e.g. via the web interface or REST API). \r\n1. Use the REST API to delete this file.\r\n1. Check the `upload/` folder and the Media Library for the behaviour described above.\r\n\r\n**Any associated stack traces or error logs**\r\n\r\nNone\r\n\n", "before_files": [{"content": "import os\nimport posixpath\nimport string\nfrom shutil import copyfileobj\n\nimport boto3\nfrom flask import current_app, redirect, send_file\nfrom flask.helpers import safe_join\nfrom werkzeug.utils import secure_filename\n\nfrom CTFd.utils import get_app_config\nfrom CTFd.utils.encoding import hexencode\n\n\nclass BaseUploader(object):\n    def __init__(self):\n        raise NotImplementedError\n\n    def store(self, fileobj, filename):\n        raise NotImplementedError\n\n    def upload(self, file_obj, filename):\n        raise NotImplementedError\n\n    def download(self, filename):\n        raise NotImplementedError\n\n    def delete(self, filename):\n        raise NotImplementedError\n\n    def sync(self):\n        raise NotImplementedError\n\n\nclass FilesystemUploader(BaseUploader):\n    def __init__(self, base_path=None):\n        super(BaseUploader, self).__init__()\n        self.base_path = base_path or current_app.config.get(\"UPLOAD_FOLDER\")\n\n    def store(self, fileobj, filename):\n        location = os.path.join(self.base_path, filename)\n        directory = os.path.dirname(location)\n\n        if not os.path.exists(directory):\n            os.makedirs(directory)\n\n        with open(location, \"wb\") as dst:\n            copyfileobj(fileobj, dst, 16384)\n\n        return filename\n\n    def upload(self, file_obj, filename):\n        if len(filename) == 0:\n            raise Exception(\"Empty filenames cannot be used\")\n\n        filename = secure_filename(filename)\n        md5hash = hexencode(os.urandom(16))\n        file_path = posixpath.join(md5hash, filename)\n\n        return self.store(file_obj, file_path)\n\n    def download(self, filename):\n        return send_file(safe_join(self.base_path, filename), as_attachment=True)\n\n    def delete(self, filename):\n        if os.path.exists(os.path.join(self.base_path, filename)):\n            os.unlink(os.path.join(self.base_path, filename))\n            return True\n        return False\n\n    def sync(self):\n        pass\n\n\nclass S3Uploader(BaseUploader):\n    def __init__(self):\n        super(BaseUploader, self).__init__()\n        self.s3 = self._get_s3_connection()\n        self.bucket = get_app_config(\"AWS_S3_BUCKET\")\n\n    def _get_s3_connection(self):\n        access_key = get_app_config(\"AWS_ACCESS_KEY_ID\")\n        secret_key = get_app_config(\"AWS_SECRET_ACCESS_KEY\")\n        endpoint = get_app_config(\"AWS_S3_ENDPOINT_URL\")\n        client = boto3.client(\n            \"s3\",\n            aws_access_key_id=access_key,\n            aws_secret_access_key=secret_key,\n            endpoint_url=endpoint,\n        )\n        return client\n\n    def _clean_filename(self, c):\n        if c in string.ascii_letters + string.digits + \"-\" + \"_\" + \".\":\n            return True\n\n    def store(self, fileobj, filename):\n        self.s3.upload_fileobj(fileobj, self.bucket, filename)\n        return filename\n\n    def upload(self, file_obj, filename):\n        filename = filter(\n            self._clean_filename, secure_filename(filename).replace(\" \", \"_\")\n        )\n        filename = \"\".join(filename)\n        if len(filename) <= 0:\n            return False\n\n        md5hash = hexencode(os.urandom(16))\n\n        dst = md5hash + \"/\" + filename\n        self.s3.upload_fileobj(file_obj, self.bucket, dst)\n        return dst\n\n    def download(self, filename):\n        key = filename\n        filename = filename.split(\"/\").pop()\n        url = self.s3.generate_presigned_url(\n            \"get_object\",\n            Params={\n                \"Bucket\": self.bucket,\n                \"Key\": key,\n                \"ResponseContentDisposition\": \"attachment; filename={}\".format(\n                    filename\n                ),\n            },\n        )\n        return redirect(url)\n\n    def delete(self, filename):\n        self.s3.delete_object(Bucket=self.bucket, Key=filename)\n        return True\n\n    def sync(self):\n        local_folder = current_app.config.get(\"UPLOAD_FOLDER\")\n        # If the bucket is empty then Contents will not be in the response\n        bucket_list = self.s3.list_objects(Bucket=self.bucket).get(\"Contents\", [])\n\n        for s3_key in bucket_list:\n            s3_object = s3_key[\"Key\"]\n            # We don't want to download any directories\n            if s3_object.endswith(\"/\") is False:\n                local_path = os.path.join(local_folder, s3_object)\n                directory = os.path.dirname(local_path)\n                if not os.path.exists(directory):\n                    os.makedirs(directory)\n\n                self.s3.download_file(self.bucket, s3_object, local_path)\n", "path": "CTFd/utils/uploads/uploaders.py"}], "after_files": [{"content": "import os\nimport posixpath\nimport string\nfrom pathlib import PurePath\nfrom shutil import copyfileobj, rmtree\n\nimport boto3\nfrom flask import current_app, redirect, send_file\nfrom flask.helpers import safe_join\nfrom werkzeug.utils import secure_filename\n\nfrom CTFd.utils import get_app_config\nfrom CTFd.utils.encoding import hexencode\n\n\nclass BaseUploader(object):\n    def __init__(self):\n        raise NotImplementedError\n\n    def store(self, fileobj, filename):\n        raise NotImplementedError\n\n    def upload(self, file_obj, filename):\n        raise NotImplementedError\n\n    def download(self, filename):\n        raise NotImplementedError\n\n    def delete(self, filename):\n        raise NotImplementedError\n\n    def sync(self):\n        raise NotImplementedError\n\n\nclass FilesystemUploader(BaseUploader):\n    def __init__(self, base_path=None):\n        super(BaseUploader, self).__init__()\n        self.base_path = base_path or current_app.config.get(\"UPLOAD_FOLDER\")\n\n    def store(self, fileobj, filename):\n        location = os.path.join(self.base_path, filename)\n        directory = os.path.dirname(location)\n\n        if not os.path.exists(directory):\n            os.makedirs(directory)\n\n        with open(location, \"wb\") as dst:\n            copyfileobj(fileobj, dst, 16384)\n\n        return filename\n\n    def upload(self, file_obj, filename):\n        if len(filename) == 0:\n            raise Exception(\"Empty filenames cannot be used\")\n\n        filename = secure_filename(filename)\n        md5hash = hexencode(os.urandom(16))\n        file_path = posixpath.join(md5hash, filename)\n\n        return self.store(file_obj, file_path)\n\n    def download(self, filename):\n        return send_file(safe_join(self.base_path, filename), as_attachment=True)\n\n    def delete(self, filename):\n        if os.path.exists(os.path.join(self.base_path, filename)):\n            file_path = PurePath(filename).parts[0]\n            rmtree(os.path.join(self.base_path, file_path))\n            return True\n        return False\n\n    def sync(self):\n        pass\n\n\nclass S3Uploader(BaseUploader):\n    def __init__(self):\n        super(BaseUploader, self).__init__()\n        self.s3 = self._get_s3_connection()\n        self.bucket = get_app_config(\"AWS_S3_BUCKET\")\n\n    def _get_s3_connection(self):\n        access_key = get_app_config(\"AWS_ACCESS_KEY_ID\")\n        secret_key = get_app_config(\"AWS_SECRET_ACCESS_KEY\")\n        endpoint = get_app_config(\"AWS_S3_ENDPOINT_URL\")\n        client = boto3.client(\n            \"s3\",\n            aws_access_key_id=access_key,\n            aws_secret_access_key=secret_key,\n            endpoint_url=endpoint,\n        )\n        return client\n\n    def _clean_filename(self, c):\n        if c in string.ascii_letters + string.digits + \"-\" + \"_\" + \".\":\n            return True\n\n    def store(self, fileobj, filename):\n        self.s3.upload_fileobj(fileobj, self.bucket, filename)\n        return filename\n\n    def upload(self, file_obj, filename):\n        filename = filter(\n            self._clean_filename, secure_filename(filename).replace(\" \", \"_\")\n        )\n        filename = \"\".join(filename)\n        if len(filename) <= 0:\n            return False\n\n        md5hash = hexencode(os.urandom(16))\n\n        dst = md5hash + \"/\" + filename\n        self.s3.upload_fileobj(file_obj, self.bucket, dst)\n        return dst\n\n    def download(self, filename):\n        key = filename\n        filename = filename.split(\"/\").pop()\n        url = self.s3.generate_presigned_url(\n            \"get_object\",\n            Params={\n                \"Bucket\": self.bucket,\n                \"Key\": key,\n                \"ResponseContentDisposition\": \"attachment; filename={}\".format(\n                    filename\n                ),\n            },\n        )\n        return redirect(url)\n\n    def delete(self, filename):\n        self.s3.delete_object(Bucket=self.bucket, Key=filename)\n        return True\n\n    def sync(self):\n        local_folder = current_app.config.get(\"UPLOAD_FOLDER\")\n        # If the bucket is empty then Contents will not be in the response\n        bucket_list = self.s3.list_objects(Bucket=self.bucket).get(\"Contents\", [])\n\n        for s3_key in bucket_list:\n            s3_object = s3_key[\"Key\"]\n            # We don't want to download any directories\n            if s3_object.endswith(\"/\") is False:\n                local_path = os.path.join(local_folder, s3_object)\n                directory = os.path.dirname(local_path)\n                if not os.path.exists(directory):\n                    os.makedirs(directory)\n\n                self.s3.download_file(self.bucket, s3_object, local_path)\n", "path": "CTFd/utils/uploads/uploaders.py"}]}