prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  Pass `Accept` header in `contrib.utils.download`
  I'm copying a comment here that I made in the [HEPData Zulip chat](https://hepdata.zulipchat.com/#narrow/stream/226203-pyhf/topic/DOIs/near/213610306) on 16th October 2020.
  
  Regarding the issue (HEPData/hepdata#162) to mint DOIs for all local resource files attached to a submission, if we do eventually get around to addressing it, we would probably redirect the DOI to a landing page for the resource file, rather than to the resource file itself (e.g. the pyhf tarball).  This would follow the DataCite [Best Practices for DOI Landing Pages](https://support.datacite.org/docs/landing-pages), e.g. "[DOIs should resolve to a landing page, not directly to the content](https://support.datacite.org/docs/landing-pages#dois-should-resolve-to-a-landing-page-not-directly-to-the-content)", which I'm currently breaking for the two manually minted DOIs.  In the issue (HEPdata/hepdata#162) I mentioned the possibility of using [DataCite Content Negotiation](https://support.datacite.org/docs/datacite-content-resolver) to redirect to the resource file itself, but the linked page now says "Custom content types are no longer supported since January 1st, 2020".  I thought maybe content negotiation could be used to return the `.tar.gz` file directly, but the intended purpose is to retrieve DOI metadata in different formats, not to provide the content itself.  In anticipation of possible future changes, I'd recommend that you use the URL directly rather than the DOI in pyhf download scripts and documentation (e.g. revert #1109).
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `src/pyhf/contrib/utils.py`
  Content:
  ```
  1 """Helper utilities for common tasks."""
  2 
  3 from urllib.parse import urlparse
  4 import tarfile
  5 from io import BytesIO
  6 import logging
  7 from pyhf import exceptions
  8 
  9 log = logging.getLogger(__name__)
  10 
  11 __all__ = ["download"]
  12 
  13 
  14 def __dir__():
  15     return __all__
  16 
  17 
  18 try:
  19     import requests
  20 
  21     def download(archive_url, output_directory, force=False, compress=False):
  22         """
  23         Download the patchset archive from the remote URL and extract it in a
  24         directory at the path given.
  25 
  26         Example:
  27 
  28             >>> from pyhf.contrib.utils import download
  29             >>> download("https://doi.org/10.17182/hepdata.90607.v3/r3", "1Lbb-likelihoods")
  30             >>> import os
  31             >>> sorted(os.listdir("1Lbb-likelihoods"))
  32             ['BkgOnly.json', 'README.md', 'patchset.json']
  33             >>> download("https://doi.org/10.17182/hepdata.90607.v3/r3", "1Lbb-likelihoods.tar.gz", compress=True)
  34             >>> import glob
  35             >>> glob.glob("1Lbb-likelihoods.tar.gz")
  36             ['1Lbb-likelihoods.tar.gz']
  37 
  38         Args:
  39             archive_url (:obj:`str`): The URL of the :class:`~pyhf.patchset.PatchSet` archive to download.
  40             output_directory (:obj:`str`): Name of the directory to unpack the archive into.
  41             force (:obj:`bool`): Force download from non-approved host. Default is ``False``.
  42             compress (:obj:`bool`): Keep the archive in a compressed ``tar.gz`` form. Default is ``False``.
  43 
  44         Raises:
  45             :class:`~pyhf.exceptions.InvalidArchiveHost`: if the provided archive host name is not known to be valid
  46         """
  47         if not force:
  48             valid_hosts = ["www.hepdata.net", "doi.org"]
  49             netloc = urlparse(archive_url).netloc
  50             if netloc not in valid_hosts:
  51                 raise exceptions.InvalidArchiveHost(
  52                     f"{netloc} is not an approved archive host: {', '.join(str(host) for host in valid_hosts)}\n"
  53                     + "To download an archive from this host use the --force option."
  54                 )
  55 
  56         with requests.get(archive_url) as response:
  57             if compress:
  58                 with open(output_directory, "wb") as archive:
  59                     archive.write(response.content)
  60             else:
  61                 with tarfile.open(
  62                     mode="r|gz", fileobj=BytesIO(response.content)
  63                 ) as archive:
  64                     archive.extractall(output_directory)
  65 
  66 
  67 except ModuleNotFoundError:
  68     log.error(
  69         "\nInstallation of the contrib extra is required to use pyhf.contrib.utils.download"
  70         + "\nPlease install with: python -m pip install pyhf[contrib]\n",
  71         exc_info=True,
  72     )
  73
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/src/pyhf/contrib/utils.py b/src/pyhf/contrib/utils.py
  --- a/src/pyhf/contrib/utils.py
  +++ b/src/pyhf/contrib/utils.py
  @@ -53,7 +53,18 @@
                       + "To download an archive from this host use the --force option."
                   )
   
  -        with requests.get(archive_url) as response:
  +        # c.f. https://github.com/scikit-hep/pyhf/issues/1491
  +        # > Use content negotiation at the landing page for the resource that
  +        # > the DOI resolves to. DataCite content negotiation is forwarding all
  +        # > requests with unknown content types to the URL registered in the
  +        # > handle system.
  +        # c.f. https://blog.datacite.org/changes-to-doi-content-negotiation/
  +        # The HEPData landing page for the resource file can check if the Accept
  +        # request HTTP header matches the content type of the resource file and
  +        # return the content directly if so.
  +        with requests.get(
  +            archive_url, headers={"Accept": "application/x-tar"}
  +        ) as response:
               if compress:
                   with open(output_directory, "wb") as archive:
                       archive.write(response.content)
  
verification_info: {"golden_diff": "diff --git a/src/pyhf/contrib/utils.py b/src/pyhf/contrib/utils.py\n--- a/src/pyhf/contrib/utils.py\n+++ b/src/pyhf/contrib/utils.py\n@@ -53,7 +53,18 @@\n                     + \"To download an archive from this host use the --force option.\"\n                 )\n \n-        with requests.get(archive_url) as response:\n+        # c.f. https://github.com/scikit-hep/pyhf/issues/1491\n+        # > Use content negotiation at the landing page for the resource that\n+        # > the DOI resolves to. DataCite content negotiation is forwarding all\n+        # > requests with unknown content types to the URL registered in the\n+        # > handle system.\n+        # c.f. https://blog.datacite.org/changes-to-doi-content-negotiation/\n+        # The HEPData landing page for the resource file can check if the Accept\n+        # request HTTP header matches the content type of the resource file and\n+        # return the content directly if so.\n+        with requests.get(\n+            archive_url, headers={\"Accept\": \"application/x-tar\"}\n+        ) as response:\n             if compress:\n                 with open(output_directory, \"wb\") as archive:\n                     archive.write(response.content)\n", "issue": "Pass `Accept` header in `contrib.utils.download`\nI'm copying a comment here that I made in the [HEPData Zulip chat](https://hepdata.zulipchat.com/#narrow/stream/226203-pyhf/topic/DOIs/near/213610306) on 16th October 2020.\r\n\r\nRegarding the issue (HEPData/hepdata#162) to mint DOIs for all local resource files attached to a submission, if we do eventually get around to addressing it, we would probably redirect the DOI to a landing page for the resource file, rather than to the resource file itself (e.g. the pyhf tarball).  This would follow the DataCite [Best Practices for DOI Landing Pages](https://support.datacite.org/docs/landing-pages), e.g. \"[DOIs should resolve to a landing page, not directly to the content](https://support.datacite.org/docs/landing-pages#dois-should-resolve-to-a-landing-page-not-directly-to-the-content)\", which I'm currently breaking for the two manually minted DOIs.  In the issue (HEPdata/hepdata#162) I mentioned the possibility of using [DataCite Content Negotiation](https://support.datacite.org/docs/datacite-content-resolver) to redirect to the resource file itself, but the linked page now says \"Custom content types are no longer supported since January 1st, 2020\".  I thought maybe content negotiation could be used to return the `.tar.gz` file directly, but the intended purpose is to retrieve DOI metadata in different formats, not to provide the content itself.  In anticipation of possible future changes, I'd recommend that you use the URL directly rather than the DOI in pyhf download scripts and documentation (e.g. revert #1109).\n", "before_files": [{"content": "\"\"\"Helper utilities for common tasks.\"\"\"\n\nfrom urllib.parse import urlparse\nimport tarfile\nfrom io import BytesIO\nimport logging\nfrom pyhf import exceptions\n\nlog = logging.getLogger(__name__)\n\n__all__ = [\"download\"]\n\n\ndef __dir__():\n    return __all__\n\n\ntry:\n    import requests\n\n    def download(archive_url, output_directory, force=False, compress=False):\n        \"\"\"\n        Download the patchset archive from the remote URL and extract it in a\n        directory at the path given.\n\n        Example:\n\n            >>> from pyhf.contrib.utils import download\n            >>> download(\"https://doi.org/10.17182/hepdata.90607.v3/r3\", \"1Lbb-likelihoods\")\n            >>> import os\n            >>> sorted(os.listdir(\"1Lbb-likelihoods\"))\n            ['BkgOnly.json', 'README.md', 'patchset.json']\n            >>> download(\"https://doi.org/10.17182/hepdata.90607.v3/r3\", \"1Lbb-likelihoods.tar.gz\", compress=True)\n            >>> import glob\n            >>> glob.glob(\"1Lbb-likelihoods.tar.gz\")\n            ['1Lbb-likelihoods.tar.gz']\n\n        Args:\n            archive_url (:obj:`str`): The URL of the :class:`~pyhf.patchset.PatchSet` archive to download.\n            output_directory (:obj:`str`): Name of the directory to unpack the archive into.\n            force (:obj:`bool`): Force download from non-approved host. Default is ``False``.\n            compress (:obj:`bool`): Keep the archive in a compressed ``tar.gz`` form. Default is ``False``.\n\n        Raises:\n            :class:`~pyhf.exceptions.InvalidArchiveHost`: if the provided archive host name is not known to be valid\n        \"\"\"\n        if not force:\n            valid_hosts = [\"www.hepdata.net\", \"doi.org\"]\n            netloc = urlparse(archive_url).netloc\n            if netloc not in valid_hosts:\n                raise exceptions.InvalidArchiveHost(\n                    f\"{netloc} is not an approved archive host: {', '.join(str(host) for host in valid_hosts)}\\n\"\n                    + \"To download an archive from this host use the --force option.\"\n                )\n\n        with requests.get(archive_url) as response:\n            if compress:\n                with open(output_directory, \"wb\") as archive:\n                    archive.write(response.content)\n            else:\n                with tarfile.open(\n                    mode=\"r|gz\", fileobj=BytesIO(response.content)\n                ) as archive:\n                    archive.extractall(output_directory)\n\n\nexcept ModuleNotFoundError:\n    log.error(\n        \"\\nInstallation of the contrib extra is required to use pyhf.contrib.utils.download\"\n        + \"\\nPlease install with: python -m pip install pyhf[contrib]\\n\",\n        exc_info=True,\n    )\n", "path": "src/pyhf/contrib/utils.py"}], "after_files": [{"content": "\"\"\"Helper utilities for common tasks.\"\"\"\n\nfrom urllib.parse import urlparse\nimport tarfile\nfrom io import BytesIO\nimport logging\nfrom pyhf import exceptions\n\nlog = logging.getLogger(__name__)\n\n__all__ = [\"download\"]\n\n\ndef __dir__():\n    return __all__\n\n\ntry:\n    import requests\n\n    def download(archive_url, output_directory, force=False, compress=False):\n        \"\"\"\n        Download the patchset archive from the remote URL and extract it in a\n        directory at the path given.\n\n        Example:\n\n            >>> from pyhf.contrib.utils import download\n            >>> download(\"https://doi.org/10.17182/hepdata.90607.v3/r3\", \"1Lbb-likelihoods\")\n            >>> import os\n            >>> sorted(os.listdir(\"1Lbb-likelihoods\"))\n            ['BkgOnly.json', 'README.md', 'patchset.json']\n            >>> download(\"https://doi.org/10.17182/hepdata.90607.v3/r3\", \"1Lbb-likelihoods.tar.gz\", compress=True)\n            >>> import glob\n            >>> glob.glob(\"1Lbb-likelihoods.tar.gz\")\n            ['1Lbb-likelihoods.tar.gz']\n\n        Args:\n            archive_url (:obj:`str`): The URL of the :class:`~pyhf.patchset.PatchSet` archive to download.\n            output_directory (:obj:`str`): Name of the directory to unpack the archive into.\n            force (:obj:`bool`): Force download from non-approved host. Default is ``False``.\n            compress (:obj:`bool`): Keep the archive in a compressed ``tar.gz`` form. Default is ``False``.\n\n        Raises:\n            :class:`~pyhf.exceptions.InvalidArchiveHost`: if the provided archive host name is not known to be valid\n        \"\"\"\n        if not force:\n            valid_hosts = [\"www.hepdata.net\", \"doi.org\"]\n            netloc = urlparse(archive_url).netloc\n            if netloc not in valid_hosts:\n                raise exceptions.InvalidArchiveHost(\n                    f\"{netloc} is not an approved archive host: {', '.join(str(host) for host in valid_hosts)}\\n\"\n                    + \"To download an archive from this host use the --force option.\"\n                )\n\n        # c.f. https://github.com/scikit-hep/pyhf/issues/1491\n        # > Use content negotiation at the landing page for the resource that\n        # > the DOI resolves to. DataCite content negotiation is forwarding all\n        # > requests with unknown content types to the URL registered in the\n        # > handle system.\n        # c.f. https://blog.datacite.org/changes-to-doi-content-negotiation/\n        # The HEPData landing page for the resource file can check if the Accept\n        # request HTTP header matches the content type of the resource file and\n        # return the content directly if so.\n        with requests.get(\n            archive_url, headers={\"Accept\": \"application/x-tar\"}\n        ) as response:\n            if compress:\n                with open(output_directory, \"wb\") as archive:\n                    archive.write(response.content)\n            else:\n                with tarfile.open(\n                    mode=\"r|gz\", fileobj=BytesIO(response.content)\n                ) as archive:\n                    archive.extractall(output_directory)\n\n\nexcept ModuleNotFoundError:\n    log.error(\n        \"\\nInstallation of the contrib extra is required to use pyhf.contrib.utils.download\"\n        + \"\\nPlease install with: python -m pip install pyhf[contrib]\\n\",\n        exc_info=True,\n    )\n", "path": "src/pyhf/contrib/utils.py"}]}