We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Setting LD_LIBRARY_PATH to be "helpful" considered harmful
(with apologies to Djikstra and everyone else who's recycled that title meme)

**TL;DR**: the library that one of my spack binaries uses depends on what other spack packages I've `module load`-ed.  **YIKES**.  See also #3926.

I was trying to understand why @JusticeForMikeBrown was having trouble building bowtie2 (see #3950) when I've built it successfully with `gcc@4.x.y`.

His problem with `gcc@4.x.y` was zlib related; I checked the package and noticed that it doesn't have a dependency on zlib.  Perhaps it should, I thought.  Wonder what zlib my "production" copy was linked against?

```
$ ldd bowtie2-align-l  | grep libz
	libz.so.1 => /blah/spack/v0.0.8/opt/spack/linux-centos7-x86_64/gcc-5.4.0/zlib-1.2.11-ec535e2ikkpl7hd4y454t3yydjqorja6/lib/libz.so.1 (0x00002aaaaaf32000)
```

That surprised me, because there's no zlib dependency in the package.

Sure enough, it's because I have something else `module load`-ed that has the side effect of adding zlib's directory to `LD_LIBRARY_PATH`.

```
$ (unset LD_LIBRARY_PATH; ldd bowtie2-align-l) | grep libz
	libz.so.1 => /lib64/libz.so.1 (0x00002aaaaaf2f000)
```

My "newer" version of CentOS has a `/lib64/libz.so.1` that includes gzbuffer (`nm` didn't help, library's stripped...):

```
$ strings /lib64/libz.so.1 | grep buffer
gzbuffer
buffer error
```

so it (probably) works for me either way.

But imagine if there were two versions of a library (perhaps something mathematical) that give different results.  Now you have a program giving different results depending on what other Spack applications are also loaded.  

**THAT** would be fun to track down (assuming you even noticed...).

W.R.T. the main problem, bowtie2 should probably have a dependency on a new-ish version of zlib, but stuff like this is why LD_LIBRARY_PATH is a slippery tool to reach for.

I'll argue that this kind of unpredictability is a bigger negative than being helpful and always setting `LD_LIBRARY_PATH`.  This comment in the docs isn't actually correct:

> Spack avoids library misconfiguration by using RPATH to link dependencies. When a user links a library or runs a program, it is tied to the dependencies it was built with, so there is no need to manipulate LD_LIBRARY_PATH at runtime.
>
> [clipped from here](http://spack.readthedocs.io/en/latest/features.html?highlight=RPATH#packages-can-peacefully-coexist)

What would happen if `LD_LIBRARY_PATH` became *opt-in*, packages that need it specify it in their package definitions?

Looking at [the list of cases where RPATH support doesn't work](http://spack.readthedocs.io/en/latest/workflows.html?highlight=LD_LIBRARY_PATH#transitive-dependencies), it seems like 1) is not relevant (I think it's referring to `PERL5LIB`, etc...) and 3) are simply bugs.  That leaves 2), python extensions.  Is `RPATH` unworkable there or just not yet working?

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `lib/spack/spack/user_environment.py`
Content:
```
1 # Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
2 # Spack Project Developers. See the top-level COPYRIGHT file for details.
3 #
4 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
5 import os
6 import sys
7 
8 import spack.build_environment
9 import spack.config
10 import spack.util.environment as environment
11 import spack.util.prefix as prefix
12 
13 #: Environment variable name Spack uses to track individually loaded packages
14 spack_loaded_hashes_var = "SPACK_LOADED_HASHES"
15 
16 
17 def prefix_inspections(platform):
18     """Get list of prefix inspections for platform
19 
20     Arguments:
21         platform (str): the name of the platform to consider. The platform
22             determines what environment variables Spack will use for some
23             inspections.
24 
25     Returns:
26         A dictionary mapping subdirectory names to lists of environment
27             variables to modify with that directory if it exists.
28     """
29     inspections = spack.config.get("modules:prefix_inspections", {})
30     if inspections:
31         return inspections
32 
33     inspections = {
34         "bin": ["PATH"],
35         "lib": ["LD_LIBRARY_PATH", "LIBRARY_PATH"],
36         "lib64": ["LD_LIBRARY_PATH", "LIBRARY_PATH"],
37         "man": ["MANPATH"],
38         "share/man": ["MANPATH"],
39         "share/aclocal": ["ACLOCAL_PATH"],
40         "include": ["CPATH"],
41         "lib/pkgconfig": ["PKG_CONFIG_PATH"],
42         "lib64/pkgconfig": ["PKG_CONFIG_PATH"],
43         "share/pkgconfig": ["PKG_CONFIG_PATH"],
44         "": ["CMAKE_PREFIX_PATH"],
45     }
46 
47     if platform == "darwin":
48         for subdir in ("lib", "lib64"):
49             inspections[subdir].append("DYLD_FALLBACK_LIBRARY_PATH")
50 
51     return inspections
52 
53 
54 def unconditional_environment_modifications(view):
55     """List of environment (shell) modifications to be processed for view.
56 
57     This list does not depend on the specs in this environment"""
58     env = environment.EnvironmentModifications()
59 
60     for subdir, vars in prefix_inspections(sys.platform).items():
61         full_subdir = os.path.join(view.root, subdir)
62         for var in vars:
63             env.prepend_path(var, full_subdir)
64 
65     return env
66 
67 
68 def environment_modifications_for_spec(spec, view=None, set_package_py_globals=True):
69     """List of environment (shell) modifications to be processed for spec.
70 
71     This list is specific to the location of the spec or its projection in
72     the view.
73 
74     Args:
75         spec (spack.spec.Spec): spec for which to list the environment modifications
76         view: view associated with the spec passed as first argument
77         set_package_py_globals (bool): whether or not to set the global variables in the
78             package.py files (this may be problematic when using buildcaches that have
79             been built on a different but compatible OS)
80     """
81     spec = spec.copy()
82     if view and not spec.external:
83         spec.prefix = prefix.Prefix(view.get_projection_for_spec(spec))
84 
85     # generic environment modifications determined by inspecting the spec
86     # prefix
87     env = environment.inspect_path(
88         spec.prefix, prefix_inspections(spec.platform), exclude=environment.is_system_path
89     )
90 
91     # Let the extendee/dependency modify their extensions/dependents
92     # before asking for package-specific modifications
93     env.extend(
94         spack.build_environment.modifications_from_dependencies(
95             spec, context="run", set_package_py_globals=set_package_py_globals
96         )
97     )
98 
99     if set_package_py_globals:
100         spack.build_environment.set_module_variables_for_package(spec.package)
101 
102     spec.package.setup_run_environment(env)
103 
104     return env
105
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/lib/spack/spack/user_environment.py b/lib/spack/spack/user_environment.py
--- a/lib/spack/spack/user_environment.py
+++ b/lib/spack/spack/user_environment.py
@@ -32,12 +32,9 @@
 
     inspections = {
         "bin": ["PATH"],
-        "lib": ["LD_LIBRARY_PATH", "LIBRARY_PATH"],
-        "lib64": ["LD_LIBRARY_PATH", "LIBRARY_PATH"],
         "man": ["MANPATH"],
         "share/man": ["MANPATH"],
         "share/aclocal": ["ACLOCAL_PATH"],
-        "include": ["CPATH"],
         "lib/pkgconfig": ["PKG_CONFIG_PATH"],
         "lib64/pkgconfig": ["PKG_CONFIG_PATH"],
         "share/pkgconfig": ["PKG_CONFIG_PATH"],
@@ -45,8 +42,8 @@
     }
 
     if platform == "darwin":
-        for subdir in ("lib", "lib64"):
-            inspections[subdir].append("DYLD_FALLBACK_LIBRARY_PATH")
+        inspections["lib"] = ["DYLD_FALLBACK_LIBRARY_PATH"]
+        inspections["lib64"] = ["DYLD_FALLBACK_LIBRARY_PATH"]
 
     return inspections

{"golden_diff": "diff --git a/lib/spack/spack/user_environment.py b/lib/spack/spack/user_environment.py\n--- a/lib/spack/spack/user_environment.py\n+++ b/lib/spack/spack/user_environment.py\n@@ -32,12 +32,9 @@\n \n     inspections = {\n         \"bin\": [\"PATH\"],\n-        \"lib\": [\"LD_LIBRARY_PATH\", \"LIBRARY_PATH\"],\n-        \"lib64\": [\"LD_LIBRARY_PATH\", \"LIBRARY_PATH\"],\n         \"man\": [\"MANPATH\"],\n         \"share/man\": [\"MANPATH\"],\n         \"share/aclocal\": [\"ACLOCAL_PATH\"],\n-        \"include\": [\"CPATH\"],\n         \"lib/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n         \"lib64/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n         \"share/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n@@ -45,8 +42,8 @@\n     }\n \n     if platform == \"darwin\":\n-        for subdir in (\"lib\", \"lib64\"):\n-            inspections[subdir].append(\"DYLD_FALLBACK_LIBRARY_PATH\")\n+        inspections[\"lib\"] = [\"DYLD_FALLBACK_LIBRARY_PATH\"]\n+        inspections[\"lib64\"] = [\"DYLD_FALLBACK_LIBRARY_PATH\"]\n \n     return inspections\n", "issue": "Setting LD_LIBRARY_PATH to be \"helpful\" considered harmful\n(with apologies to Djikstra and everyone else who's recycled that title meme)\r\n\r\n**TL;DR**: the library that one of my spack binaries uses depends on what other spack packages I've `module load`-ed.  **YIKES**.  See also #3926.\r\n\r\nI was trying to understand why @JusticeForMikeBrown was having trouble building bowtie2 (see #3950) when I've built it successfully with `gcc@4.x.y`.\r\n\r\nHis problem with `gcc@4.x.y` was zlib related; I checked the package and noticed that it doesn't have a dependency on zlib.  Perhaps it should, I thought.  Wonder what zlib my \"production\" copy was linked against?\r\n\r\n```\r\n$ ldd bowtie2-align-l  | grep libz\r\n\tlibz.so.1 => /blah/spack/v0.0.8/opt/spack/linux-centos7-x86_64/gcc-5.4.0/zlib-1.2.11-ec535e2ikkpl7hd4y454t3yydjqorja6/lib/libz.so.1 (0x00002aaaaaf32000)\r\n```\r\n\r\nThat surprised me, because there's no zlib dependency in the package.\r\n\r\nSure enough, it's because I have something else `module load`-ed that has the side effect of adding zlib's directory to `LD_LIBRARY_PATH`.\r\n\r\n```\r\n$ (unset LD_LIBRARY_PATH; ldd bowtie2-align-l) | grep libz\r\n\tlibz.so.1 => /lib64/libz.so.1 (0x00002aaaaaf2f000)\r\n```\r\n\r\nMy \"newer\" version of CentOS has a `/lib64/libz.so.1` that includes gzbuffer (`nm` didn't help, library's stripped...):\r\n\r\n```\r\n$ strings /lib64/libz.so.1 | grep buffer\r\ngzbuffer\r\nbuffer error\r\n```\r\n\r\nso it (probably) works for me either way.\r\n\r\nBut imagine if there were two versions of a library (perhaps something mathematical) that give different results.  Now you have a program giving different results depending on what other Spack applications are also loaded.  \r\n\r\n**THAT** would be fun to track down (assuming you even noticed...).\r\n\r\nW.R.T. the main problem, bowtie2 should probably have a dependency on a new-ish version of zlib, but stuff like this is why LD_LIBRARY_PATH is a slippery tool to reach for.\r\n\r\nI'll argue that this kind of unpredictability is a bigger negative than being helpful and always setting `LD_LIBRARY_PATH`.  This comment in the docs isn't actually correct:\r\n\r\n> Spack avoids library misconfiguration by using RPATH to link dependencies. When a user links a library or runs a program, it is tied to the dependencies it was built with, so there is no need to manipulate LD_LIBRARY_PATH at runtime.\r\n>\r\n> [clipped from here](http://spack.readthedocs.io/en/latest/features.html?highlight=RPATH#packages-can-peacefully-coexist)\r\n\r\nWhat would happen if `LD_LIBRARY_PATH` became *opt-in*, packages that need it specify it in their package definitions?\r\n\r\nLooking at [the list of cases where RPATH support doesn't work](http://spack.readthedocs.io/en/latest/workflows.html?highlight=LD_LIBRARY_PATH#transitive-dependencies), it seems like 1) is not relevant (I think it's referring to `PERL5LIB`, etc...) and 3) are simply bugs.  That leaves 2), python extensions.  Is `RPATH` unworkable there or just not yet working?\n", "before_files": [{"content": "# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other\n# Spack Project Developers. See the top-level COPYRIGHT file for details.\n#\n# SPDX-License-Identifier: (Apache-2.0 OR MIT)\nimport os\nimport sys\n\nimport spack.build_environment\nimport spack.config\nimport spack.util.environment as environment\nimport spack.util.prefix as prefix\n\n#: Environment variable name Spack uses to track individually loaded packages\nspack_loaded_hashes_var = \"SPACK_LOADED_HASHES\"\n\n\ndef prefix_inspections(platform):\n    \"\"\"Get list of prefix inspections for platform\n\n    Arguments:\n        platform (str): the name of the platform to consider. The platform\n            determines what environment variables Spack will use for some\n            inspections.\n\n    Returns:\n        A dictionary mapping subdirectory names to lists of environment\n            variables to modify with that directory if it exists.\n    \"\"\"\n    inspections = spack.config.get(\"modules:prefix_inspections\", {})\n    if inspections:\n        return inspections\n\n    inspections = {\n        \"bin\": [\"PATH\"],\n        \"lib\": [\"LD_LIBRARY_PATH\", \"LIBRARY_PATH\"],\n        \"lib64\": [\"LD_LIBRARY_PATH\", \"LIBRARY_PATH\"],\n        \"man\": [\"MANPATH\"],\n        \"share/man\": [\"MANPATH\"],\n        \"share/aclocal\": [\"ACLOCAL_PATH\"],\n        \"include\": [\"CPATH\"],\n        \"lib/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"lib64/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"share/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"\": [\"CMAKE_PREFIX_PATH\"],\n    }\n\n    if platform == \"darwin\":\n        for subdir in (\"lib\", \"lib64\"):\n            inspections[subdir].append(\"DYLD_FALLBACK_LIBRARY_PATH\")\n\n    return inspections\n\n\ndef unconditional_environment_modifications(view):\n    \"\"\"List of environment (shell) modifications to be processed for view.\n\n    This list does not depend on the specs in this environment\"\"\"\n    env = environment.EnvironmentModifications()\n\n    for subdir, vars in prefix_inspections(sys.platform).items():\n        full_subdir = os.path.join(view.root, subdir)\n        for var in vars:\n            env.prepend_path(var, full_subdir)\n\n    return env\n\n\ndef environment_modifications_for_spec(spec, view=None, set_package_py_globals=True):\n    \"\"\"List of environment (shell) modifications to be processed for spec.\n\n    This list is specific to the location of the spec or its projection in\n    the view.\n\n    Args:\n        spec (spack.spec.Spec): spec for which to list the environment modifications\n        view: view associated with the spec passed as first argument\n        set_package_py_globals (bool): whether or not to set the global variables in the\n            package.py files (this may be problematic when using buildcaches that have\n            been built on a different but compatible OS)\n    \"\"\"\n    spec = spec.copy()\n    if view and not spec.external:\n        spec.prefix = prefix.Prefix(view.get_projection_for_spec(spec))\n\n    # generic environment modifications determined by inspecting the spec\n    # prefix\n    env = environment.inspect_path(\n        spec.prefix, prefix_inspections(spec.platform), exclude=environment.is_system_path\n    )\n\n    # Let the extendee/dependency modify their extensions/dependents\n    # before asking for package-specific modifications\n    env.extend(\n        spack.build_environment.modifications_from_dependencies(\n            spec, context=\"run\", set_package_py_globals=set_package_py_globals\n        )\n    )\n\n    if set_package_py_globals:\n        spack.build_environment.set_module_variables_for_package(spec.package)\n\n    spec.package.setup_run_environment(env)\n\n    return env\n", "path": "lib/spack/spack/user_environment.py"}], "after_files": [{"content": "# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other\n# Spack Project Developers. See the top-level COPYRIGHT file for details.\n#\n# SPDX-License-Identifier: (Apache-2.0 OR MIT)\nimport os\nimport sys\n\nimport spack.build_environment\nimport spack.config\nimport spack.util.environment as environment\nimport spack.util.prefix as prefix\n\n#: Environment variable name Spack uses to track individually loaded packages\nspack_loaded_hashes_var = \"SPACK_LOADED_HASHES\"\n\n\ndef prefix_inspections(platform):\n    \"\"\"Get list of prefix inspections for platform\n\n    Arguments:\n        platform (str): the name of the platform to consider. The platform\n            determines what environment variables Spack will use for some\n            inspections.\n\n    Returns:\n        A dictionary mapping subdirectory names to lists of environment\n            variables to modify with that directory if it exists.\n    \"\"\"\n    inspections = spack.config.get(\"modules:prefix_inspections\", {})\n    if inspections:\n        return inspections\n\n    inspections = {\n        \"bin\": [\"PATH\"],\n        \"man\": [\"MANPATH\"],\n        \"share/man\": [\"MANPATH\"],\n        \"share/aclocal\": [\"ACLOCAL_PATH\"],\n        \"lib/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"lib64/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"share/pkgconfig\": [\"PKG_CONFIG_PATH\"],\n        \"\": [\"CMAKE_PREFIX_PATH\"],\n    }\n\n    if platform == \"darwin\":\n        inspections[\"lib\"] = [\"DYLD_FALLBACK_LIBRARY_PATH\"]\n        inspections[\"lib64\"] = [\"DYLD_FALLBACK_LIBRARY_PATH\"]\n\n    return inspections\n\n\ndef unconditional_environment_modifications(view):\n    \"\"\"List of environment (shell) modifications to be processed for view.\n\n    This list does not depend on the specs in this environment\"\"\"\n    env = environment.EnvironmentModifications()\n\n    for subdir, vars in prefix_inspections(sys.platform).items():\n        full_subdir = os.path.join(view.root, subdir)\n        for var in vars:\n            env.prepend_path(var, full_subdir)\n\n    return env\n\n\ndef environment_modifications_for_spec(spec, view=None, set_package_py_globals=True):\n    \"\"\"List of environment (shell) modifications to be processed for spec.\n\n    This list is specific to the location of the spec or its projection in\n    the view.\n\n    Args:\n        spec (spack.spec.Spec): spec for which to list the environment modifications\n        view: view associated with the spec passed as first argument\n        set_package_py_globals (bool): whether or not to set the global variables in the\n            package.py files (this may be problematic when using buildcaches that have\n            been built on a different but compatible OS)\n    \"\"\"\n    spec = spec.copy()\n    if view and not spec.external:\n        spec.prefix = prefix.Prefix(view.get_projection_for_spec(spec))\n\n    # generic environment modifications determined by inspecting the spec\n    # prefix\n    env = environment.inspect_path(\n        spec.prefix, prefix_inspections(spec.platform), exclude=environment.is_system_path\n    )\n\n    # Let the extendee/dependency modify their extensions/dependents\n    # before asking for package-specific modifications\n    env.extend(\n        spack.build_environment.modifications_from_dependencies(\n            spec, context=\"run\", set_package_py_globals=set_package_py_globals\n        )\n    )\n\n    if set_package_py_globals:\n        spack.build_environment.set_module_variables_for_package(spec.package)\n\n    spec.package.setup_run_environment(env)\n\n    return env\n", "path": "lib/spack/spack/user_environment.py"}]}