We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[BUG] Healthcheck fails when using a custom runtime dir
### What docker image(s) are you using?

scipy-notebook (but applies to all images based on the `base-notebook` image)

### Host OS system

RHEL 8.0

### Host architecture

x86_64

### What Docker command are you running?

The following command DOES work as expected (default runtime dir):
```
docker run --rm -p 8888:8888 --name jupyter quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh
```

The following command does NOT work as expected (customized runtime dir):
```
docker run --rm -p 8888:8888 --name jupyter -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh
```

### How to Reproduce the problem?

1. Start the Jupyter container using the commands above.
2. In another terminal, run the healtcheck script: `docker exec jupyter /etc/jupyter/docker_healthcheck.py`
3. Observe the healthcheck script failing due to server state JSON file(s) not being found.

### Command output

```bash session
$ docker run --rm -p 8888:8888 --name jupyter quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh
$ docker exec jupyter /etc/jupyter/docker_healthcheck.py
b'{"version": "2.12.1"}'

$ docker run --rm -p 8888:8888 --name jupyter -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh
$ docker exec jupyter /etc/jupyter/docker_healthcheck.py
Traceback (most recent call last):
  File "/etc/jupyter/docker_healthcheck.py", line 14, in <module>
    json_file = next(runtime_dir.glob("*server-*.json"))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration
```


### Expected behavior

Healthcheck script to not fail, e.g. to display `b'{"version": "2.12.1"}'`, even with a customized runtime dir.

### Actual behavior

The healthcheck script fails because it cannot find server state JSON files in the hard-coded default runtime dir.

### Anything else?

The problem is that the `/etc/jupyter/docker_healthcheck.py` healtcheck script hard-codes the default runtime directory to search for server JSON state files as below:
https://github.com/jupyter/docker-stacks/blob/fcb20a914ed20e44a96053caf43eef6e12fb4c04/images/base-notebook/docker_healthcheck.py#L13

When this directory is customized for example via `JUPYTER_RUNTIME_DIR`, then the healthcheck script does not work.

The actual problem is when deploying Jupyter containers as services.
The Jupyter images have a default healthcheck configured as below:
https://github.com/jupyter/docker-stacks/blob/fcb20a914ed20e44a96053caf43eef6e12fb4c04/images/base-notebook/Dockerfile#L66-L70

When the healthcheck fails due to a custom runtime dir, the service is restarted continuously.

I think the healthcheck script should use the output of `jupyter --runtime-dir` which respects customizations:
```
$ docker run --rm -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 jupyter --runtime-dir
/home/jovyan/custom-runtime
```

If you agree with the above, I can send a PR with this fix.

### Latest Docker version

- [X] I've updated my Docker version to the latest available, and the issue persists

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `images/base-notebook/docker_healthcheck.py`
Content:
```
1 #!/usr/bin/env python3
2 # Copyright (c) Jupyter Development Team.
3 # Distributed under the terms of the Modified BSD License.
4 import json
5 import os
6 from pathlib import Path
7 
8 import requests
9 
10 # Several operations below deliberately don't check for possible errors
11 # As this is a healthcheck, it should succeed or raise an exception on error
12 
13 runtime_dir = Path("/home/") / os.environ["NB_USER"] / ".local/share/jupyter/runtime/"
14 json_file = next(runtime_dir.glob("*server-*.json"))
15 
16 url = json.loads(json_file.read_bytes())["url"]
17 url = url + "api"
18 
19 proxies = {
20     "http": "",
21     "https": "",
22 }
23 
24 r = requests.get(url, proxies=proxies, verify=False)  # request without SSL verification
25 r.raise_for_status()
26 print(r.content)
27
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/images/base-notebook/docker_healthcheck.py b/images/base-notebook/docker_healthcheck.py
--- a/images/base-notebook/docker_healthcheck.py
+++ b/images/base-notebook/docker_healthcheck.py
@@ -3,6 +3,7 @@
 # Distributed under the terms of the Modified BSD License.
 import json
 import os
+import subprocess
 from pathlib import Path
 
 import requests
@@ -10,7 +11,19 @@
 # Several operations below deliberately don't check for possible errors
 # As this is a healthcheck, it should succeed or raise an exception on error
 
-runtime_dir = Path("/home/") / os.environ["NB_USER"] / ".local/share/jupyter/runtime/"
+# Docker runs healtchecks using an exec
+# It uses the default user configured when running the image: root for the case of a custom NB_USER or jovyan for the case of the default image user.
+# We manually change HOME to make `jupyter --runtime-dir` report a correct path
+# More information: <https://github.com/jupyter/docker-stacks/pull/2074#issuecomment-1879778409>
+result = subprocess.run(
+    ["jupyter", "--runtime-dir"],
+    check=True,
+    capture_output=True,
+    text=True,
+    env=dict(os.environ) | {"HOME": "/home/" + os.environ["NB_USER"]},
+)
+runtime_dir = Path(result.stdout.rstrip())
+
 json_file = next(runtime_dir.glob("*server-*.json"))
 
 url = json.loads(json_file.read_bytes())["url"]

{"golden_diff": "diff --git a/images/base-notebook/docker_healthcheck.py b/images/base-notebook/docker_healthcheck.py\n--- a/images/base-notebook/docker_healthcheck.py\n+++ b/images/base-notebook/docker_healthcheck.py\n@@ -3,6 +3,7 @@\n # Distributed under the terms of the Modified BSD License.\n import json\n import os\n+import subprocess\n from pathlib import Path\n \n import requests\n@@ -10,7 +11,19 @@\n # Several operations below deliberately don't check for possible errors\n # As this is a healthcheck, it should succeed or raise an exception on error\n \n-runtime_dir = Path(\"/home/\") / os.environ[\"NB_USER\"] / \".local/share/jupyter/runtime/\"\n+# Docker runs healtchecks using an exec\n+# It uses the default user configured when running the image: root for the case of a custom NB_USER or jovyan for the case of the default image user.\n+# We manually change HOME to make `jupyter --runtime-dir` report a correct path\n+# More information: <https://github.com/jupyter/docker-stacks/pull/2074#issuecomment-1879778409>\n+result = subprocess.run(\n+    [\"jupyter\", \"--runtime-dir\"],\n+    check=True,\n+    capture_output=True,\n+    text=True,\n+    env=dict(os.environ) | {\"HOME\": \"/home/\" + os.environ[\"NB_USER\"]},\n+)\n+runtime_dir = Path(result.stdout.rstrip())\n+\n json_file = next(runtime_dir.glob(\"*server-*.json\"))\n \n url = json.loads(json_file.read_bytes())[\"url\"]\n", "issue": "[BUG] Healthcheck fails when using a custom runtime dir\n### What docker image(s) are you using?\r\n\r\nscipy-notebook (but applies to all images based on the `base-notebook` image)\r\n\r\n### Host OS system\r\n\r\nRHEL 8.0\r\n\r\n### Host architecture\r\n\r\nx86_64\r\n\r\n### What Docker command are you running?\r\n\r\nThe following command DOES work as expected (default runtime dir):\r\n```\r\ndocker run --rm -p 8888:8888 --name jupyter quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh\r\n```\r\n\r\nThe following command does NOT work as expected (customized runtime dir):\r\n```\r\ndocker run --rm -p 8888:8888 --name jupyter -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh\r\n```\r\n\r\n### How to Reproduce the problem?\r\n\r\n1. Start the Jupyter container using the commands above.\r\n2. In another terminal, run the healtcheck script: `docker exec jupyter /etc/jupyter/docker_healthcheck.py`\r\n3. Observe the healthcheck script failing due to server state JSON file(s) not being found.\r\n\r\n### Command output\r\n\r\n```bash session\r\n$ docker run --rm -p 8888:8888 --name jupyter quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh\r\n$ docker exec jupyter /etc/jupyter/docker_healthcheck.py\r\nb'{\"version\": \"2.12.1\"}'\r\n\r\n$ docker run --rm -p 8888:8888 --name jupyter -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 start-notebook.sh\r\n$ docker exec jupyter /etc/jupyter/docker_healthcheck.py\r\nTraceback (most recent call last):\r\n  File \"/etc/jupyter/docker_healthcheck.py\", line 14, in <module>\r\n    json_file = next(runtime_dir.glob(\"*server-*.json\"))\r\n                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\nStopIteration\r\n```\r\n\r\n\r\n### Expected behavior\r\n\r\nHealthcheck script to not fail, e.g. to display `b'{\"version\": \"2.12.1\"}'`, even with a customized runtime dir.\r\n\r\n### Actual behavior\r\n\r\nThe healthcheck script fails because it cannot find server state JSON files in the hard-coded default runtime dir.\r\n\r\n### Anything else?\r\n\r\nThe problem is that the `/etc/jupyter/docker_healthcheck.py` healtcheck script hard-codes the default runtime directory to search for server JSON state files as below:\r\nhttps://github.com/jupyter/docker-stacks/blob/fcb20a914ed20e44a96053caf43eef6e12fb4c04/images/base-notebook/docker_healthcheck.py#L13\r\n\r\nWhen this directory is customized for example via `JUPYTER_RUNTIME_DIR`, then the healthcheck script does not work.\r\n\r\nThe actual problem is when deploying Jupyter containers as services.\r\nThe Jupyter images have a default healthcheck configured as below:\r\nhttps://github.com/jupyter/docker-stacks/blob/fcb20a914ed20e44a96053caf43eef6e12fb4c04/images/base-notebook/Dockerfile#L66-L70\r\n\r\nWhen the healthcheck fails due to a custom runtime dir, the service is restarted continuously.\r\n\r\nI think the healthcheck script should use the output of `jupyter --runtime-dir` which respects customizations:\r\n```\r\n$ docker run --rm -e JUPYTER_RUNTIME_DIR=/home/jovyan/custom-runtime quay.io/jupyter/scipy-notebook:2023-12-25 jupyter --runtime-dir\r\n/home/jovyan/custom-runtime\r\n```\r\n\r\nIf you agree with the above, I can send a PR with this fix.\r\n\r\n### Latest Docker version\r\n\r\n- [X] I've updated my Docker version to the latest available, and the issue persists\n", "before_files": [{"content": "#!/usr/bin/env python3\n# Copyright (c) Jupyter Development Team.\n# Distributed under the terms of the Modified BSD License.\nimport json\nimport os\nfrom pathlib import Path\n\nimport requests\n\n# Several operations below deliberately don't check for possible errors\n# As this is a healthcheck, it should succeed or raise an exception on error\n\nruntime_dir = Path(\"/home/\") / os.environ[\"NB_USER\"] / \".local/share/jupyter/runtime/\"\njson_file = next(runtime_dir.glob(\"*server-*.json\"))\n\nurl = json.loads(json_file.read_bytes())[\"url\"]\nurl = url + \"api\"\n\nproxies = {\n    \"http\": \"\",\n    \"https\": \"\",\n}\n\nr = requests.get(url, proxies=proxies, verify=False)  # request without SSL verification\nr.raise_for_status()\nprint(r.content)\n", "path": "images/base-notebook/docker_healthcheck.py"}], "after_files": [{"content": "#!/usr/bin/env python3\n# Copyright (c) Jupyter Development Team.\n# Distributed under the terms of the Modified BSD License.\nimport json\nimport os\nimport subprocess\nfrom pathlib import Path\n\nimport requests\n\n# Several operations below deliberately don't check for possible errors\n# As this is a healthcheck, it should succeed or raise an exception on error\n\n# Docker runs healtchecks using an exec\n# It uses the default user configured when running the image: root for the case of a custom NB_USER or jovyan for the case of the default image user.\n# We manually change HOME to make `jupyter --runtime-dir` report a correct path\n# More information: <https://github.com/jupyter/docker-stacks/pull/2074#issuecomment-1879778409>\nresult = subprocess.run(\n    [\"jupyter\", \"--runtime-dir\"],\n    check=True,\n    capture_output=True,\n    text=True,\n    env=dict(os.environ) | {\"HOME\": \"/home/\" + os.environ[\"NB_USER\"]},\n)\nruntime_dir = Path(result.stdout.rstrip())\n\njson_file = next(runtime_dir.glob(\"*server-*.json\"))\n\nurl = json.loads(json_file.read_bytes())[\"url\"]\nurl = url + \"api\"\n\nproxies = {\n    \"http\": \"\",\n    \"https\": \"\",\n}\n\nr = requests.get(url, proxies=proxies, verify=False)  # request without SSL verification\nr.raise_for_status()\nprint(r.content)\n", "path": "images/base-notebook/docker_healthcheck.py"}]}