prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  Add lxml dependency
  ### Checklist
  
  - [X] This is a feature request and not a different kind of issue
  - [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)
  - [X] [I have checked the list of open and recently closed plugin requests](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22feature+request%22)
  
  ### Description
  
  Streamlink should finally switch to a proper HTML/XML parser for extracting data instead of using cheap regex workarounds which don't work properly. I've already commented on this issue last year:
  https://github.com/streamlink/streamlink/issues/3241#issuecomment-706486239
  
  The reason why I'm suggesting this again right now is that I was trying to fix the deutschewelle plugin (https://dw.com) yesterday and ran into issues with the `itertags` utility method, which is based on simple regexes for iterating HTML nodes and their attributes+body. `itertags` for example does not work with nested nodes, which makes adding ridiculous custom regexes necessary. Just take a look at this madness:
  https://github.com/streamlink/streamlink/blob/3668770d608f0fab54d40a46acd6720a97f63775/src/streamlink/plugins/deutschewelle.py#L18-L29
  
  With `lxml` (https://lxml.de/), HTML page contents can be parsed and the data extracted via XPath queries and/or the respective API methods. The methods are similar to python's native `xml.etree.ElementTree`, which itself is considered too slow and unsafe in certain cases. I am by no means an expert regarding python's standard library though, so if someone has better insight here, please share. In regards to packaging, this lib is available on basically every packaging system and adding it as a dependency here only has benefits.
  
  I'd suggest that we add `lxml` as a dependency now and start using it for extracting data from HTML documents. The validation schema methods could be improved for this as well. There's also the `parse_xml` utility method, which is currently based on the native module.
  
  Comments?
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `setup.py`
  Content:
  ```
  1 #!/usr/bin/env python
  2 import codecs
  3 from os import environ, path
  4 from sys import argv, path as sys_path
  5 
  6 from setuptools import find_packages, setup
  7 
  8 import versioneer
  9 
  10 
  11 data_files = []
  12 deps = [
  13     "requests>=2.26.0,<3.0",
  14     "isodate",
  15     "websocket-client>=0.58.0",
  16     # Support for SOCKS proxies
  17     "PySocks!=1.5.7,>=1.5.6",
  18 ]
  19 
  20 # for encrypted streams
  21 if environ.get("STREAMLINK_USE_PYCRYPTO"):
  22     deps.append("pycrypto")
  23 else:
  24     # this version of pycryptodome is known to work and has a Windows wheel for py2.7, py3.3-3.6
  25     deps.append("pycryptodome>=3.4.3,<4")
  26 
  27 # for localization
  28 if environ.get("STREAMLINK_USE_PYCOUNTRY"):
  29     deps.append("pycountry")
  30 else:
  31     deps.append("iso-639")
  32     deps.append("iso3166")
  33 
  34 # When we build an egg for the Win32 bootstrap we don"t want dependency
  35 # information built into it.
  36 if environ.get("NO_DEPS"):
  37     deps = []
  38 
  39 this_directory = path.abspath(path.dirname(__file__))
  40 srcdir = path.join(this_directory, "src/")
  41 sys_path.insert(0, srcdir)
  42 
  43 with codecs.open(path.join(this_directory, "README.md"), 'r', "utf8") as f:
  44     long_description = f.read()
  45 
  46 
  47 def is_wheel_for_windows():
  48     if "bdist_wheel" in argv:
  49         names = ["win32", "win-amd64", "cygwin"]
  50         length = len(argv)
  51         for pos in range(argv.index("bdist_wheel") + 1, length):
  52             if argv[pos] == "--plat-name" and pos + 1 < length:
  53                 return argv[pos + 1] in names
  54             elif argv[pos][:12] == "--plat-name=":
  55                 return argv[pos][12:] in names
  56     return False
  57 
  58 
  59 entry_points = {
  60     "console_scripts": ["streamlink=streamlink_cli.main:main"]
  61 }
  62 
  63 if is_wheel_for_windows():
  64     entry_points["gui_scripts"] = ["streamlinkw=streamlink_cli.main:main"]
  65 
  66 
  67 additional_files = [
  68     ("share/man/man1", ["docs/_build/man/streamlink.1"])
  69 ]
  70 
  71 for destdir, srcfiles in additional_files:
  72     files = []
  73     for srcfile in srcfiles:
  74         if path.exists(srcfile):
  75             files.append(srcfile)
  76     if files:
  77         data_files.append((destdir, files))
  78 
  79 
  80 setup(name="streamlink",
  81       version=versioneer.get_version(),
  82       cmdclass=versioneer.get_cmdclass(),
  83       description="Streamlink is a command-line utility that extracts streams "
  84                   "from various services and pipes them into a video player of "
  85                   "choice.",
  86       long_description=long_description,
  87       long_description_content_type="text/markdown",
  88       url="https://github.com/streamlink/streamlink",
  89       project_urls={
  90           "Documentation": "https://streamlink.github.io/",
  91           "Tracker": "https://github.com/streamlink/streamlink/issues",
  92           "Source": "https://github.com/streamlink/streamlink",
  93           "Funding": "https://opencollective.com/streamlink"
  94       },
  95       author="Streamlink",
  96       # temp until we have a mailing list / global email
  97       author_email="streamlink@protonmail.com",
  98       license="Simplified BSD",
  99       packages=find_packages("src"),
  100       package_dir={"": "src"},
  101       package_data={"streamlink.plugins": [".removed"]},
  102       entry_points=entry_points,
  103       data_files=data_files,
  104       install_requires=deps,
  105       test_suite="tests",
  106       python_requires=">=3.6, <4",
  107       classifiers=["Development Status :: 5 - Production/Stable",
  108                    "License :: OSI Approved :: BSD License",
  109                    "Environment :: Console",
  110                    "Intended Audience :: End Users/Desktop",
  111                    "Operating System :: POSIX",
  112                    "Operating System :: Microsoft :: Windows",
  113                    "Operating System :: MacOS",
  114                    "Programming Language :: Python :: 3",
  115                    "Programming Language :: Python :: 3 :: Only",
  116                    "Programming Language :: Python :: 3.6",
  117                    "Programming Language :: Python :: 3.7",
  118                    "Programming Language :: Python :: 3.8",
  119                    "Programming Language :: Python :: 3.9",
  120                    "Topic :: Internet :: WWW/HTTP",
  121                    "Topic :: Multimedia :: Sound/Audio",
  122                    "Topic :: Multimedia :: Video",
  123                    "Topic :: Utilities"])
  124
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/setup.py b/setup.py
  --- a/setup.py
  +++ b/setup.py
  @@ -12,6 +12,7 @@
   deps = [
       "requests>=2.26.0,<3.0",
       "isodate",
  +    "lxml>=4.6.3",
       "websocket-client>=0.58.0",
       # Support for SOCKS proxies
       "PySocks!=1.5.7,>=1.5.6",
  
verification_info: {"golden_diff": "diff --git a/setup.py b/setup.py\n--- a/setup.py\n+++ b/setup.py\n@@ -12,6 +12,7 @@\n deps = [\n     \"requests>=2.26.0,<3.0\",\n     \"isodate\",\n+    \"lxml>=4.6.3\",\n     \"websocket-client>=0.58.0\",\n     # Support for SOCKS proxies\n     \"PySocks!=1.5.7,>=1.5.6\",\n", "issue": "Add lxml dependency\n### Checklist\n\n- [X] This is a feature request and not a different kind of issue\n- [X] [I have read the contribution guidelines](https://github.com/streamlink/streamlink/blob/master/CONTRIBUTING.md#contributing-to-streamlink)\n- [X] [I have checked the list of open and recently closed plugin requests](https://github.com/streamlink/streamlink/issues?q=is%3Aissue+label%3A%22feature+request%22)\n\n### Description\n\nStreamlink should finally switch to a proper HTML/XML parser for extracting data instead of using cheap regex workarounds which don't work properly. I've already commented on this issue last year:\r\nhttps://github.com/streamlink/streamlink/issues/3241#issuecomment-706486239\r\n\r\nThe reason why I'm suggesting this again right now is that I was trying to fix the deutschewelle plugin (https://dw.com) yesterday and ran into issues with the `itertags` utility method, which is based on simple regexes for iterating HTML nodes and their attributes+body. `itertags` for example does not work with nested nodes, which makes adding ridiculous custom regexes necessary. Just take a look at this madness:\r\nhttps://github.com/streamlink/streamlink/blob/3668770d608f0fab54d40a46acd6720a97f63775/src/streamlink/plugins/deutschewelle.py#L18-L29\r\n\r\nWith `lxml` (https://lxml.de/), HTML page contents can be parsed and the data extracted via XPath queries and/or the respective API methods. The methods are similar to python's native `xml.etree.ElementTree`, which itself is considered too slow and unsafe in certain cases. I am by no means an expert regarding python's standard library though, so if someone has better insight here, please share. In regards to packaging, this lib is available on basically every packaging system and adding it as a dependency here only has benefits.\r\n\r\nI'd suggest that we add `lxml` as a dependency now and start using it for extracting data from HTML documents. The validation schema methods could be improved for this as well. There's also the `parse_xml` utility method, which is currently based on the native module.\r\n\r\nComments?\n", "before_files": [{"content": "#!/usr/bin/env python\nimport codecs\nfrom os import environ, path\nfrom sys import argv, path as sys_path\n\nfrom setuptools import find_packages, setup\n\nimport versioneer\n\n\ndata_files = []\ndeps = [\n    \"requests>=2.26.0,<3.0\",\n    \"isodate\",\n    \"websocket-client>=0.58.0\",\n    # Support for SOCKS proxies\n    \"PySocks!=1.5.7,>=1.5.6\",\n]\n\n# for encrypted streams\nif environ.get(\"STREAMLINK_USE_PYCRYPTO\"):\n    deps.append(\"pycrypto\")\nelse:\n    # this version of pycryptodome is known to work and has a Windows wheel for py2.7, py3.3-3.6\n    deps.append(\"pycryptodome>=3.4.3,<4\")\n\n# for localization\nif environ.get(\"STREAMLINK_USE_PYCOUNTRY\"):\n    deps.append(\"pycountry\")\nelse:\n    deps.append(\"iso-639\")\n    deps.append(\"iso3166\")\n\n# When we build an egg for the Win32 bootstrap we don\"t want dependency\n# information built into it.\nif environ.get(\"NO_DEPS\"):\n    deps = []\n\nthis_directory = path.abspath(path.dirname(__file__))\nsrcdir = path.join(this_directory, \"src/\")\nsys_path.insert(0, srcdir)\n\nwith codecs.open(path.join(this_directory, \"README.md\"), 'r', \"utf8\") as f:\n    long_description = f.read()\n\n\ndef is_wheel_for_windows():\n    if \"bdist_wheel\" in argv:\n        names = [\"win32\", \"win-amd64\", \"cygwin\"]\n        length = len(argv)\n        for pos in range(argv.index(\"bdist_wheel\") + 1, length):\n            if argv[pos] == \"--plat-name\" and pos + 1 < length:\n                return argv[pos + 1] in names\n            elif argv[pos][:12] == \"--plat-name=\":\n                return argv[pos][12:] in names\n    return False\n\n\nentry_points = {\n    \"console_scripts\": [\"streamlink=streamlink_cli.main:main\"]\n}\n\nif is_wheel_for_windows():\n    entry_points[\"gui_scripts\"] = [\"streamlinkw=streamlink_cli.main:main\"]\n\n\nadditional_files = [\n    (\"share/man/man1\", [\"docs/_build/man/streamlink.1\"])\n]\n\nfor destdir, srcfiles in additional_files:\n    files = []\n    for srcfile in srcfiles:\n        if path.exists(srcfile):\n            files.append(srcfile)\n    if files:\n        data_files.append((destdir, files))\n\n\nsetup(name=\"streamlink\",\n      version=versioneer.get_version(),\n      cmdclass=versioneer.get_cmdclass(),\n      description=\"Streamlink is a command-line utility that extracts streams \"\n                  \"from various services and pipes them into a video player of \"\n                  \"choice.\",\n      long_description=long_description,\n      long_description_content_type=\"text/markdown\",\n      url=\"https://github.com/streamlink/streamlink\",\n      project_urls={\n          \"Documentation\": \"https://streamlink.github.io/\",\n          \"Tracker\": \"https://github.com/streamlink/streamlink/issues\",\n          \"Source\": \"https://github.com/streamlink/streamlink\",\n          \"Funding\": \"https://opencollective.com/streamlink\"\n      },\n      author=\"Streamlink\",\n      # temp until we have a mailing list / global email\n      author_email=\"streamlink@protonmail.com\",\n      license=\"Simplified BSD\",\n      packages=find_packages(\"src\"),\n      package_dir={\"\": \"src\"},\n      package_data={\"streamlink.plugins\": [\".removed\"]},\n      entry_points=entry_points,\n      data_files=data_files,\n      install_requires=deps,\n      test_suite=\"tests\",\n      python_requires=\">=3.6, <4\",\n      classifiers=[\"Development Status :: 5 - Production/Stable\",\n                   \"License :: OSI Approved :: BSD License\",\n                   \"Environment :: Console\",\n                   \"Intended Audience :: End Users/Desktop\",\n                   \"Operating System :: POSIX\",\n                   \"Operating System :: Microsoft :: Windows\",\n                   \"Operating System :: MacOS\",\n                   \"Programming Language :: Python :: 3\",\n                   \"Programming Language :: Python :: 3 :: Only\",\n                   \"Programming Language :: Python :: 3.6\",\n                   \"Programming Language :: Python :: 3.7\",\n                   \"Programming Language :: Python :: 3.8\",\n                   \"Programming Language :: Python :: 3.9\",\n                   \"Topic :: Internet :: WWW/HTTP\",\n                   \"Topic :: Multimedia :: Sound/Audio\",\n                   \"Topic :: Multimedia :: Video\",\n                   \"Topic :: Utilities\"])\n", "path": "setup.py"}], "after_files": [{"content": "#!/usr/bin/env python\nimport codecs\nfrom os import environ, path\nfrom sys import argv, path as sys_path\n\nfrom setuptools import find_packages, setup\n\nimport versioneer\n\n\ndata_files = []\ndeps = [\n    \"requests>=2.26.0,<3.0\",\n    \"isodate\",\n    \"lxml>=4.6.3\",\n    \"websocket-client>=0.58.0\",\n    # Support for SOCKS proxies\n    \"PySocks!=1.5.7,>=1.5.6\",\n]\n\n# for encrypted streams\nif environ.get(\"STREAMLINK_USE_PYCRYPTO\"):\n    deps.append(\"pycrypto\")\nelse:\n    # this version of pycryptodome is known to work and has a Windows wheel for py2.7, py3.3-3.6\n    deps.append(\"pycryptodome>=3.4.3,<4\")\n\n# for localization\nif environ.get(\"STREAMLINK_USE_PYCOUNTRY\"):\n    deps.append(\"pycountry\")\nelse:\n    deps.append(\"iso-639\")\n    deps.append(\"iso3166\")\n\n# When we build an egg for the Win32 bootstrap we don\"t want dependency\n# information built into it.\nif environ.get(\"NO_DEPS\"):\n    deps = []\n\nthis_directory = path.abspath(path.dirname(__file__))\nsrcdir = path.join(this_directory, \"src/\")\nsys_path.insert(0, srcdir)\n\nwith codecs.open(path.join(this_directory, \"README.md\"), 'r', \"utf8\") as f:\n    long_description = f.read()\n\n\ndef is_wheel_for_windows():\n    if \"bdist_wheel\" in argv:\n        names = [\"win32\", \"win-amd64\", \"cygwin\"]\n        length = len(argv)\n        for pos in range(argv.index(\"bdist_wheel\") + 1, length):\n            if argv[pos] == \"--plat-name\" and pos + 1 < length:\n                return argv[pos + 1] in names\n            elif argv[pos][:12] == \"--plat-name=\":\n                return argv[pos][12:] in names\n    return False\n\n\nentry_points = {\n    \"console_scripts\": [\"streamlink=streamlink_cli.main:main\"]\n}\n\nif is_wheel_for_windows():\n    entry_points[\"gui_scripts\"] = [\"streamlinkw=streamlink_cli.main:main\"]\n\n\nadditional_files = [\n    (\"share/man/man1\", [\"docs/_build/man/streamlink.1\"])\n]\n\nfor destdir, srcfiles in additional_files:\n    files = []\n    for srcfile in srcfiles:\n        if path.exists(srcfile):\n            files.append(srcfile)\n    if files:\n        data_files.append((destdir, files))\n\n\nsetup(name=\"streamlink\",\n      version=versioneer.get_version(),\n      cmdclass=versioneer.get_cmdclass(),\n      description=\"Streamlink is a command-line utility that extracts streams \"\n                  \"from various services and pipes them into a video player of \"\n                  \"choice.\",\n      long_description=long_description,\n      long_description_content_type=\"text/markdown\",\n      url=\"https://github.com/streamlink/streamlink\",\n      project_urls={\n          \"Documentation\": \"https://streamlink.github.io/\",\n          \"Tracker\": \"https://github.com/streamlink/streamlink/issues\",\n          \"Source\": \"https://github.com/streamlink/streamlink\",\n          \"Funding\": \"https://opencollective.com/streamlink\"\n      },\n      author=\"Streamlink\",\n      # temp until we have a mailing list / global email\n      author_email=\"streamlink@protonmail.com\",\n      license=\"Simplified BSD\",\n      packages=find_packages(\"src\"),\n      package_dir={\"\": \"src\"},\n      package_data={\"streamlink.plugins\": [\".removed\"]},\n      entry_points=entry_points,\n      data_files=data_files,\n      install_requires=deps,\n      test_suite=\"tests\",\n      python_requires=\">=3.6, <4\",\n      classifiers=[\"Development Status :: 5 - Production/Stable\",\n                   \"License :: OSI Approved :: BSD License\",\n                   \"Environment :: Console\",\n                   \"Intended Audience :: End Users/Desktop\",\n                   \"Operating System :: POSIX\",\n                   \"Operating System :: Microsoft :: Windows\",\n                   \"Operating System :: MacOS\",\n                   \"Programming Language :: Python :: 3\",\n                   \"Programming Language :: Python :: 3 :: Only\",\n                   \"Programming Language :: Python :: 3.6\",\n                   \"Programming Language :: Python :: 3.7\",\n                   \"Programming Language :: Python :: 3.8\",\n                   \"Programming Language :: Python :: 3.9\",\n                   \"Topic :: Internet :: WWW/HTTP\",\n                   \"Topic :: Multimedia :: Sound/Audio\",\n                   \"Topic :: Multimedia :: Video\",\n                   \"Topic :: Utilities\"])\n", "path": "setup.py"}]}