We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Converting from half precision to `bfloat16` in Pearson correlation coefficient cause numerical errors
## 🐛 Bug

The lack of half-precision op support on CPUs has lead to the decision to convert to `bfloat16` in the calculation of Pearson correlation coefficient. (#1813)

However this leads to broadcasting errors when the values are (meaningfully) small.


### To Reproduce

```py
>>> import torch
>>> a = torch.rand((32,), dtype=torch.half)
>>> b = torch.rand((32,), dtype=torch.half)
>>> from torchmetrics.functional import pearson_corrcoef
>>> pearson_corrcoef(a, b)
tensor(0.2240)
>>> pearson_corrcoef(a * 1e-1, b * 1e-1)
tensor(0.2251)
>>> pearson_corrcoef(a * 1e-2, b * 1e-2)
tensor(0.2213)
>>> pearson_corrcoef(a * 1e-3, b * 1e-3)
tensor(0.)
>>> pearson_corrcoef(a * 1e-4, b * 1e-4)
tensor(nan)
```

Which generally makes sense since there are only 5 exponential bits in `float16`, and some are lost in the operations. However this is not obvious during debugging since 0.001 is not *that* small...

### Expected behavior

A warning when the dynamic range is problematic, such as [what SciPy does](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.pearsonr.html#scipy-stats-pearsonr).

Or there may be some way to preserve more precision?

### Environment

- TorchMetrics version (pypi v1.0.1):
- Python & PyTorch Version (3.10, 2.0.1):
- Any other relevant information such as OS (tested on Linux_x64 and macOS_arm64):


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `src/torchmetrics/functional/regression/pearson.py`
Content:
```
1 # Copyright The Lightning team.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 from typing import Tuple
15 
16 import torch
17 from torch import Tensor
18 
19 from torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs
20 from torchmetrics.utilities.checks import _check_same_shape
21 
22 
23 def _pearson_corrcoef_update(
24     preds: Tensor,
25     target: Tensor,
26     mean_x: Tensor,
27     mean_y: Tensor,
28     var_x: Tensor,
29     var_y: Tensor,
30     corr_xy: Tensor,
31     n_prior: Tensor,
32     num_outputs: int,
33 ) -> Tuple[Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]:
34     """Update and returns variables required to compute Pearson Correlation Coefficient.
35 
36     Check for same shape of input tensors.
37 
38     Args:
39         preds: estimated scores
40         target: ground truth scores
41         mean_x: current mean estimate of x tensor
42         mean_y: current mean estimate of y tensor
43         var_x: current variance estimate of x tensor
44         var_y: current variance estimate of y tensor
45         corr_xy: current covariance estimate between x and y tensor
46         n_prior: current number of observed observations
47         num_outputs: Number of outputs in multioutput setting
48 
49     """
50     # Data checking
51     _check_same_shape(preds, target)
52     _check_data_shape_to_num_outputs(preds, target, num_outputs)
53     cond = n_prior.mean() > 0
54 
55     n_obs = preds.shape[0]
56     if cond:
57         mx_new = (n_prior * mean_x + preds.sum(0)) / (n_prior + n_obs)
58         my_new = (n_prior * mean_y + target.sum(0)) / (n_prior + n_obs)
59     else:
60         mx_new = preds.mean(0)
61         my_new = target.mean(0)
62 
63     n_prior += n_obs
64 
65     if cond:
66         var_x += ((preds - mx_new) * (preds - mean_x)).sum(0)
67         var_y += ((target - my_new) * (target - mean_y)).sum(0)
68 
69     else:
70         var_x += preds.var(0) * (n_obs - 1)
71         var_y += target.var(0) * (n_obs - 1)
72     corr_xy += ((preds - mx_new) * (target - mean_y)).sum(0)
73     mean_x = mx_new
74     mean_y = my_new
75 
76     return mean_x, mean_y, var_x, var_y, corr_xy, n_prior
77 
78 
79 def _pearson_corrcoef_compute(
80     var_x: Tensor,
81     var_y: Tensor,
82     corr_xy: Tensor,
83     nb: Tensor,
84 ) -> Tensor:
85     """Compute the final pearson correlation based on accumulated statistics.
86 
87     Args:
88         var_x: variance estimate of x tensor
89         var_y: variance estimate of y tensor
90         corr_xy: covariance estimate between x and y tensor
91         nb: number of observations
92 
93     """
94     var_x /= nb - 1
95     var_y /= nb - 1
96     corr_xy /= nb - 1
97     # if var_x, var_y is float16 and on cpu, make it bfloat16 as sqrt is not supported for float16
98     # on cpu, remove this after https://github.com/pytorch/pytorch/issues/54774 is fixed
99     if var_x.dtype == torch.float16 and var_x.device == torch.device("cpu"):
100         var_x = var_x.bfloat16()
101         var_y = var_y.bfloat16()
102 
103     corrcoef = (corr_xy / (var_x * var_y).sqrt()).squeeze()
104     return torch.clamp(corrcoef, -1.0, 1.0)
105 
106 
107 def pearson_corrcoef(preds: Tensor, target: Tensor) -> Tensor:
108     """Compute pearson correlation coefficient.
109 
110     Args:
111         preds: estimated scores
112         target: ground truth scores
113 
114     Example (single output regression):
115         >>> from torchmetrics.functional.regression import pearson_corrcoef
116         >>> target = torch.tensor([3, -0.5, 2, 7])
117         >>> preds = torch.tensor([2.5, 0.0, 2, 8])
118         >>> pearson_corrcoef(preds, target)
119         tensor(0.9849)
120 
121     Example (multi output regression):
122         >>> from torchmetrics.functional.regression import pearson_corrcoef
123         >>> target = torch.tensor([[3, -0.5], [2, 7]])
124         >>> preds = torch.tensor([[2.5, 0.0], [2, 8]])
125         >>> pearson_corrcoef(preds, target)
126         tensor([1., 1.])
127 
128     """
129     d = preds.shape[1] if preds.ndim == 2 else 1
130     _temp = torch.zeros(d, dtype=preds.dtype, device=preds.device)
131     mean_x, mean_y, var_x = _temp.clone(), _temp.clone(), _temp.clone()
132     var_y, corr_xy, nb = _temp.clone(), _temp.clone(), _temp.clone()
133     _, _, var_x, var_y, corr_xy, nb = _pearson_corrcoef_update(
134         preds, target, mean_x, mean_y, var_x, var_y, corr_xy, nb, num_outputs=1 if preds.ndim == 1 else preds.shape[-1]
135     )
136     return _pearson_corrcoef_compute(var_x, var_y, corr_xy, nb)
137
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/src/torchmetrics/functional/regression/pearson.py b/src/torchmetrics/functional/regression/pearson.py
--- a/src/torchmetrics/functional/regression/pearson.py
+++ b/src/torchmetrics/functional/regression/pearson.py
@@ -11,12 +11,14 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+import math
 from typing import Tuple
 
 import torch
 from torch import Tensor
 
 from torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs
+from torchmetrics.utilities import rank_zero_warn
 from torchmetrics.utilities.checks import _check_same_shape
 
 
@@ -100,6 +102,15 @@
         var_x = var_x.bfloat16()
         var_y = var_y.bfloat16()
 
+    bound = math.sqrt(torch.finfo(var_x.dtype).eps)
+    if (var_x < bound).any() or (var_y < bound).any():
+        rank_zero_warn(
+            "The variance of predictions or target is close to zero. This can cause instability in Pearson correlation"
+            "coefficient, leading to wrong results. Consider re-scaling the input if possible or computing using a"
+            f"larger dtype (currently using {var_x.dtype}).",
+            UserWarning,
+        )
+
     corrcoef = (corr_xy / (var_x * var_y).sqrt()).squeeze()
     return torch.clamp(corrcoef, -1.0, 1.0)

{"golden_diff": "diff --git a/src/torchmetrics/functional/regression/pearson.py b/src/torchmetrics/functional/regression/pearson.py\n--- a/src/torchmetrics/functional/regression/pearson.py\n+++ b/src/torchmetrics/functional/regression/pearson.py\n@@ -11,12 +11,14 @@\n # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n # See the License for the specific language governing permissions and\n # limitations under the License.\n+import math\n from typing import Tuple\n \n import torch\n from torch import Tensor\n \n from torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs\n+from torchmetrics.utilities import rank_zero_warn\n from torchmetrics.utilities.checks import _check_same_shape\n \n \n@@ -100,6 +102,15 @@\n         var_x = var_x.bfloat16()\n         var_y = var_y.bfloat16()\n \n+    bound = math.sqrt(torch.finfo(var_x.dtype).eps)\n+    if (var_x < bound).any() or (var_y < bound).any():\n+        rank_zero_warn(\n+            \"The variance of predictions or target is close to zero. This can cause instability in Pearson correlation\"\n+            \"coefficient, leading to wrong results. Consider re-scaling the input if possible or computing using a\"\n+            f\"larger dtype (currently using {var_x.dtype}).\",\n+            UserWarning,\n+        )\n+\n     corrcoef = (corr_xy / (var_x * var_y).sqrt()).squeeze()\n     return torch.clamp(corrcoef, -1.0, 1.0)\n", "issue": "Converting from half precision to `bfloat16` in Pearson correlation coefficient cause numerical errors\n## \ud83d\udc1b Bug\r\n\r\nThe lack of half-precision op support on CPUs has lead to the decision to convert to `bfloat16` in the calculation of Pearson correlation coefficient. (#1813)\r\n\r\nHowever this leads to broadcasting errors when the values are (meaningfully) small.\r\n\r\n\r\n### To Reproduce\r\n\r\n```py\r\n>>> import torch\r\n>>> a = torch.rand((32,), dtype=torch.half)\r\n>>> b = torch.rand((32,), dtype=torch.half)\r\n>>> from torchmetrics.functional import pearson_corrcoef\r\n>>> pearson_corrcoef(a, b)\r\ntensor(0.2240)\r\n>>> pearson_corrcoef(a * 1e-1, b * 1e-1)\r\ntensor(0.2251)\r\n>>> pearson_corrcoef(a * 1e-2, b * 1e-2)\r\ntensor(0.2213)\r\n>>> pearson_corrcoef(a * 1e-3, b * 1e-3)\r\ntensor(0.)\r\n>>> pearson_corrcoef(a * 1e-4, b * 1e-4)\r\ntensor(nan)\r\n```\r\n\r\nWhich generally makes sense since there are only 5 exponential bits in `float16`, and some are lost in the operations. However this is not obvious during debugging since 0.001 is not *that* small...\r\n\r\n### Expected behavior\r\n\r\nA warning when the dynamic range is problematic, such as [what SciPy does](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.pearsonr.html#scipy-stats-pearsonr).\r\n\r\nOr there may be some way to preserve more precision?\r\n\r\n### Environment\r\n\r\n- TorchMetrics version (pypi v1.0.1):\r\n- Python & PyTorch Version (3.10, 2.0.1):\r\n- Any other relevant information such as OS (tested on Linux_x64 and macOS_arm64):\r\n\n", "before_files": [{"content": "# Copyright The Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import Tuple\n\nimport torch\nfrom torch import Tensor\n\nfrom torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs\nfrom torchmetrics.utilities.checks import _check_same_shape\n\n\ndef _pearson_corrcoef_update(\n    preds: Tensor,\n    target: Tensor,\n    mean_x: Tensor,\n    mean_y: Tensor,\n    var_x: Tensor,\n    var_y: Tensor,\n    corr_xy: Tensor,\n    n_prior: Tensor,\n    num_outputs: int,\n) -> Tuple[Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]:\n    \"\"\"Update and returns variables required to compute Pearson Correlation Coefficient.\n\n    Check for same shape of input tensors.\n\n    Args:\n        preds: estimated scores\n        target: ground truth scores\n        mean_x: current mean estimate of x tensor\n        mean_y: current mean estimate of y tensor\n        var_x: current variance estimate of x tensor\n        var_y: current variance estimate of y tensor\n        corr_xy: current covariance estimate between x and y tensor\n        n_prior: current number of observed observations\n        num_outputs: Number of outputs in multioutput setting\n\n    \"\"\"\n    # Data checking\n    _check_same_shape(preds, target)\n    _check_data_shape_to_num_outputs(preds, target, num_outputs)\n    cond = n_prior.mean() > 0\n\n    n_obs = preds.shape[0]\n    if cond:\n        mx_new = (n_prior * mean_x + preds.sum(0)) / (n_prior + n_obs)\n        my_new = (n_prior * mean_y + target.sum(0)) / (n_prior + n_obs)\n    else:\n        mx_new = preds.mean(0)\n        my_new = target.mean(0)\n\n    n_prior += n_obs\n\n    if cond:\n        var_x += ((preds - mx_new) * (preds - mean_x)).sum(0)\n        var_y += ((target - my_new) * (target - mean_y)).sum(0)\n\n    else:\n        var_x += preds.var(0) * (n_obs - 1)\n        var_y += target.var(0) * (n_obs - 1)\n    corr_xy += ((preds - mx_new) * (target - mean_y)).sum(0)\n    mean_x = mx_new\n    mean_y = my_new\n\n    return mean_x, mean_y, var_x, var_y, corr_xy, n_prior\n\n\ndef _pearson_corrcoef_compute(\n    var_x: Tensor,\n    var_y: Tensor,\n    corr_xy: Tensor,\n    nb: Tensor,\n) -> Tensor:\n    \"\"\"Compute the final pearson correlation based on accumulated statistics.\n\n    Args:\n        var_x: variance estimate of x tensor\n        var_y: variance estimate of y tensor\n        corr_xy: covariance estimate between x and y tensor\n        nb: number of observations\n\n    \"\"\"\n    var_x /= nb - 1\n    var_y /= nb - 1\n    corr_xy /= nb - 1\n    # if var_x, var_y is float16 and on cpu, make it bfloat16 as sqrt is not supported for float16\n    # on cpu, remove this after https://github.com/pytorch/pytorch/issues/54774 is fixed\n    if var_x.dtype == torch.float16 and var_x.device == torch.device(\"cpu\"):\n        var_x = var_x.bfloat16()\n        var_y = var_y.bfloat16()\n\n    corrcoef = (corr_xy / (var_x * var_y).sqrt()).squeeze()\n    return torch.clamp(corrcoef, -1.0, 1.0)\n\n\ndef pearson_corrcoef(preds: Tensor, target: Tensor) -> Tensor:\n    \"\"\"Compute pearson correlation coefficient.\n\n    Args:\n        preds: estimated scores\n        target: ground truth scores\n\n    Example (single output regression):\n        >>> from torchmetrics.functional.regression import pearson_corrcoef\n        >>> target = torch.tensor([3, -0.5, 2, 7])\n        >>> preds = torch.tensor([2.5, 0.0, 2, 8])\n        >>> pearson_corrcoef(preds, target)\n        tensor(0.9849)\n\n    Example (multi output regression):\n        >>> from torchmetrics.functional.regression import pearson_corrcoef\n        >>> target = torch.tensor([[3, -0.5], [2, 7]])\n        >>> preds = torch.tensor([[2.5, 0.0], [2, 8]])\n        >>> pearson_corrcoef(preds, target)\n        tensor([1., 1.])\n\n    \"\"\"\n    d = preds.shape[1] if preds.ndim == 2 else 1\n    _temp = torch.zeros(d, dtype=preds.dtype, device=preds.device)\n    mean_x, mean_y, var_x = _temp.clone(), _temp.clone(), _temp.clone()\n    var_y, corr_xy, nb = _temp.clone(), _temp.clone(), _temp.clone()\n    _, _, var_x, var_y, corr_xy, nb = _pearson_corrcoef_update(\n        preds, target, mean_x, mean_y, var_x, var_y, corr_xy, nb, num_outputs=1 if preds.ndim == 1 else preds.shape[-1]\n    )\n    return _pearson_corrcoef_compute(var_x, var_y, corr_xy, nb)\n", "path": "src/torchmetrics/functional/regression/pearson.py"}], "after_files": [{"content": "# Copyright The Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport math\nfrom typing import Tuple\n\nimport torch\nfrom torch import Tensor\n\nfrom torchmetrics.functional.regression.utils import _check_data_shape_to_num_outputs\nfrom torchmetrics.utilities import rank_zero_warn\nfrom torchmetrics.utilities.checks import _check_same_shape\n\n\ndef _pearson_corrcoef_update(\n    preds: Tensor,\n    target: Tensor,\n    mean_x: Tensor,\n    mean_y: Tensor,\n    var_x: Tensor,\n    var_y: Tensor,\n    corr_xy: Tensor,\n    n_prior: Tensor,\n    num_outputs: int,\n) -> Tuple[Tensor, Tensor, Tensor, Tensor, Tensor, Tensor]:\n    \"\"\"Update and returns variables required to compute Pearson Correlation Coefficient.\n\n    Check for same shape of input tensors.\n\n    Args:\n        preds: estimated scores\n        target: ground truth scores\n        mean_x: current mean estimate of x tensor\n        mean_y: current mean estimate of y tensor\n        var_x: current variance estimate of x tensor\n        var_y: current variance estimate of y tensor\n        corr_xy: current covariance estimate between x and y tensor\n        n_prior: current number of observed observations\n        num_outputs: Number of outputs in multioutput setting\n\n    \"\"\"\n    # Data checking\n    _check_same_shape(preds, target)\n    _check_data_shape_to_num_outputs(preds, target, num_outputs)\n    cond = n_prior.mean() > 0\n\n    n_obs = preds.shape[0]\n    if cond:\n        mx_new = (n_prior * mean_x + preds.sum(0)) / (n_prior + n_obs)\n        my_new = (n_prior * mean_y + target.sum(0)) / (n_prior + n_obs)\n    else:\n        mx_new = preds.mean(0)\n        my_new = target.mean(0)\n\n    n_prior += n_obs\n\n    if cond:\n        var_x += ((preds - mx_new) * (preds - mean_x)).sum(0)\n        var_y += ((target - my_new) * (target - mean_y)).sum(0)\n\n    else:\n        var_x += preds.var(0) * (n_obs - 1)\n        var_y += target.var(0) * (n_obs - 1)\n    corr_xy += ((preds - mx_new) * (target - mean_y)).sum(0)\n    mean_x = mx_new\n    mean_y = my_new\n\n    return mean_x, mean_y, var_x, var_y, corr_xy, n_prior\n\n\ndef _pearson_corrcoef_compute(\n    var_x: Tensor,\n    var_y: Tensor,\n    corr_xy: Tensor,\n    nb: Tensor,\n) -> Tensor:\n    \"\"\"Compute the final pearson correlation based on accumulated statistics.\n\n    Args:\n        var_x: variance estimate of x tensor\n        var_y: variance estimate of y tensor\n        corr_xy: covariance estimate between x and y tensor\n        nb: number of observations\n\n    \"\"\"\n    var_x /= nb - 1\n    var_y /= nb - 1\n    corr_xy /= nb - 1\n    # if var_x, var_y is float16 and on cpu, make it bfloat16 as sqrt is not supported for float16\n    # on cpu, remove this after https://github.com/pytorch/pytorch/issues/54774 is fixed\n    if var_x.dtype == torch.float16 and var_x.device == torch.device(\"cpu\"):\n        var_x = var_x.bfloat16()\n        var_y = var_y.bfloat16()\n\n    bound = math.sqrt(torch.finfo(var_x.dtype).eps)\n    if (var_x < bound).any() or (var_y < bound).any():\n        rank_zero_warn(\n            \"The variance of predictions or target is close to zero. This can cause instability in Pearson correlation\"\n            \"coefficient, leading to wrong results. Consider re-scaling the input if possible or computing using a\"\n            f\"larger dtype (currently using {var_x.dtype}).\",\n            UserWarning,\n        )\n\n    corrcoef = (corr_xy / (var_x * var_y).sqrt()).squeeze()\n    return torch.clamp(corrcoef, -1.0, 1.0)\n\n\ndef pearson_corrcoef(preds: Tensor, target: Tensor) -> Tensor:\n    \"\"\"Compute pearson correlation coefficient.\n\n    Args:\n        preds: estimated scores\n        target: ground truth scores\n\n    Example (single output regression):\n        >>> from torchmetrics.functional.regression import pearson_corrcoef\n        >>> target = torch.tensor([3, -0.5, 2, 7])\n        >>> preds = torch.tensor([2.5, 0.0, 2, 8])\n        >>> pearson_corrcoef(preds, target)\n        tensor(0.9849)\n\n    Example (multi output regression):\n        >>> from torchmetrics.functional.regression import pearson_corrcoef\n        >>> target = torch.tensor([[3, -0.5], [2, 7]])\n        >>> preds = torch.tensor([[2.5, 0.0], [2, 8]])\n        >>> pearson_corrcoef(preds, target)\n        tensor([1., 1.])\n\n    \"\"\"\n    d = preds.shape[1] if preds.ndim == 2 else 1\n    _temp = torch.zeros(d, dtype=preds.dtype, device=preds.device)\n    mean_x, mean_y, var_x = _temp.clone(), _temp.clone(), _temp.clone()\n    var_y, corr_xy, nb = _temp.clone(), _temp.clone(), _temp.clone()\n    _, _, var_x, var_y, corr_xy, nb = _pearson_corrcoef_update(\n        preds, target, mean_x, mean_y, var_x, var_y, corr_xy, nb, num_outputs=1 if preds.ndim == 1 else preds.shape[-1]\n    )\n    return _pearson_corrcoef_compute(var_x, var_y, corr_xy, nb)\n", "path": "src/torchmetrics/functional/regression/pearson.py"}]}