We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
Incorrect results with ignore_index in IoU
## 🐛 Bug

The ignore_index option in IoU does not behave as it should

### To Reproduce

Run the above code

#### Code sample

```python
import torch
import torchmetrics

iou = torchmetrics.IoU(3, ignore_index=0)
print(iou(torch.tensor([[1,2,1,1]]), torch.tensor([[2,1,1,0]])))
print(iou(torch.tensor([[1,2,1,0]]), torch.tensor([[2,1,1,0]])))
> tensor(0.1250)
tensor(0.1667)
```


### Expected behavior

Predicting a '0' here should not modify the score, both should give 0.1667

### Environment

 - PyTorch Version (e.g., 1.0): 1.8.1
 - OS (e.g., Linux): Linux & Windows
 - How you installed PyTorch (`conda`, `pip`, source): both
 - Build command you used (if compiling from source):
 - Python version: 3.8
 - CUDA/cuDNN version: 
 - GPU models and configuration:
 - Any other relevant information: 

### Additional context

The ignore_index is implemented like [this](https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/functional/classification/iou.py#L39):
```
if ignore_index is not None and 0 <= ignore_index < num_classes:
    scores = torch.cat([
        scores[:ignore_index],
        scores[ignore_index + 1:],
    ])
```
This is incorrect as it does not provide the right union value. Instead, the confmat should be stripped of its ignore_index column and line.

This can be done with this line provided it is put before the computation of the intersection and union:
`confmat = confmat[torch.arange(confmat.size(0)) != ignore_index].T[[ignore_index != torch.arange(confmat.size(0))]].T`


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `torchmetrics/functional/classification/iou.py`
Content:
```
1 # Copyright The PyTorch Lightning team.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
4 # you may not use this file except in compliance with the License.
5 # You may obtain a copy of the License at
6 #
7 #     http://www.apache.org/licenses/LICENSE-2.0
8 #
9 # Unless required by applicable law or agreed to in writing, software
10 # distributed under the License is distributed on an "AS IS" BASIS,
11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
14 from typing import Optional
15 
16 import torch
17 from torch import Tensor
18 
19 from torchmetrics.functional.classification.confusion_matrix import _confusion_matrix_update
20 from torchmetrics.utilities.data import get_num_classes
21 from torchmetrics.utilities.distributed import reduce
22 
23 
24 def _iou_from_confmat(
25     confmat: Tensor,
26     num_classes: int,
27     ignore_index: Optional[int] = None,
28     absent_score: float = 0.0,
29     reduction: str = 'elementwise_mean',
30 ) -> Tensor:
31     intersection = torch.diag(confmat)
32     union = confmat.sum(0) + confmat.sum(1) - intersection
33 
34     # If this class is absent in both target AND pred (union == 0), then use the absent_score for this class.
35     scores = intersection.float() / union.float()
36     scores[union == 0] = absent_score
37 
38     # Remove the ignored class index from the scores.
39     if ignore_index is not None and 0 <= ignore_index < num_classes:
40         scores = torch.cat([
41             scores[:ignore_index],
42             scores[ignore_index + 1:],
43         ])
44     return reduce(scores, reduction=reduction)
45 
46 
47 def iou(
48     preds: Tensor,
49     target: Tensor,
50     ignore_index: Optional[int] = None,
51     absent_score: float = 0.0,
52     threshold: float = 0.5,
53     num_classes: Optional[int] = None,
54     reduction: str = 'elementwise_mean',
55 ) -> Tensor:
56     r"""
57     Computes `Intersection over union, or Jaccard index calculation <https://en.wikipedia.org/wiki/Jaccard_index>`_:
58 
59     .. math:: J(A,B) = \frac{|A\cap B|}{|A\cup B|}
60 
61     Where: :math:`A` and :math:`B` are both tensors of the same size,
62     containing integer class values. They may be subject to conversion from
63     input data (see description below).
64 
65     Note that it is different from box IoU.
66 
67     If preds and target are the same shape and preds is a float tensor, we use the ``self.threshold`` argument
68     to convert into integer labels. This is the case for binary and multi-label probabilities.
69 
70     If pred has an extra dimension as in the case of multi-class scores we
71     perform an argmax on ``dim=1``.
72 
73     Args:
74         preds: tensor containing predictions from model (probabilities, or labels) with shape ``[N, d1, d2, ...]``
75         target: tensor containing ground truth labels with shape ``[N, d1, d2, ...]``
76         ignore_index: optional int specifying a target class to ignore. If given,
77             this class index does not contribute to the returned score, regardless
78             of reduction method. Has no effect if given an int that is not in the
79             range [0, num_classes-1], where num_classes is either given or derived
80             from pred and target. By default, no index is ignored, and all classes are used.
81         absent_score: score to use for an individual class, if no instances of
82             the class index were present in `pred` AND no instances of the class
83             index were present in `target`. For example, if we have 3 classes,
84             [0, 0] for `pred`, and [0, 2] for `target`, then class 1 would be
85             assigned the `absent_score`.
86         threshold:
87             Threshold value for binary or multi-label probabilities. default: 0.5
88         num_classes:
89             Optionally specify the number of classes
90         reduction: a method to reduce metric score over labels.
91 
92             - ``'elementwise_mean'``: takes the mean (default)
93             - ``'sum'``: takes the sum
94             - ``'none'``: no reduction will be applied
95 
96     Return:
97         IoU score : Tensor containing single value if reduction is
98         'elementwise_mean', or number of classes if reduction is 'none'
99 
100     Example:
101         >>> from torchmetrics.functional import iou
102         >>> target = torch.randint(0, 2, (10, 25, 25))
103         >>> pred = torch.tensor(target)
104         >>> pred[2:5, 7:13, 9:15] = 1 - pred[2:5, 7:13, 9:15]
105         >>> iou(pred, target)
106         tensor(0.9660)
107     """
108 
109     num_classes = get_num_classes(preds=preds, target=target, num_classes=num_classes)
110     confmat = _confusion_matrix_update(preds, target, num_classes, threshold)
111     return _iou_from_confmat(confmat, num_classes, ignore_index, absent_score, reduction)
112
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/torchmetrics/functional/classification/iou.py b/torchmetrics/functional/classification/iou.py
--- a/torchmetrics/functional/classification/iou.py
+++ b/torchmetrics/functional/classification/iou.py
@@ -28,6 +28,11 @@
     absent_score: float = 0.0,
     reduction: str = 'elementwise_mean',
 ) -> Tensor:
+
+    # Remove the ignored class index from the scores.
+    if ignore_index is not None and 0 <= ignore_index < num_classes:
+        confmat[ignore_index] = 0.
+
     intersection = torch.diag(confmat)
     union = confmat.sum(0) + confmat.sum(1) - intersection
 
@@ -35,12 +40,12 @@
     scores = intersection.float() / union.float()
     scores[union == 0] = absent_score
 
-    # Remove the ignored class index from the scores.
     if ignore_index is not None and 0 <= ignore_index < num_classes:
         scores = torch.cat([
             scores[:ignore_index],
             scores[ignore_index + 1:],
         ])
+
     return reduce(scores, reduction=reduction)

{"golden_diff": "diff --git a/torchmetrics/functional/classification/iou.py b/torchmetrics/functional/classification/iou.py\n--- a/torchmetrics/functional/classification/iou.py\n+++ b/torchmetrics/functional/classification/iou.py\n@@ -28,6 +28,11 @@\n     absent_score: float = 0.0,\n     reduction: str = 'elementwise_mean',\n ) -> Tensor:\n+\n+    # Remove the ignored class index from the scores.\n+    if ignore_index is not None and 0 <= ignore_index < num_classes:\n+        confmat[ignore_index] = 0.\n+\n     intersection = torch.diag(confmat)\n     union = confmat.sum(0) + confmat.sum(1) - intersection\n \n@@ -35,12 +40,12 @@\n     scores = intersection.float() / union.float()\n     scores[union == 0] = absent_score\n \n-    # Remove the ignored class index from the scores.\n     if ignore_index is not None and 0 <= ignore_index < num_classes:\n         scores = torch.cat([\n             scores[:ignore_index],\n             scores[ignore_index + 1:],\n         ])\n+\n     return reduce(scores, reduction=reduction)\n", "issue": "Incorrect results with ignore_index in IoU\n## \ud83d\udc1b Bug\r\n\r\nThe ignore_index option in IoU does not behave as it should\r\n\r\n### To Reproduce\r\n\r\nRun the above code\r\n\r\n#### Code sample\r\n\r\n```python\r\nimport torch\r\nimport torchmetrics\r\n\r\niou = torchmetrics.IoU(3, ignore_index=0)\r\nprint(iou(torch.tensor([[1,2,1,1]]), torch.tensor([[2,1,1,0]])))\r\nprint(iou(torch.tensor([[1,2,1,0]]), torch.tensor([[2,1,1,0]])))\r\n> tensor(0.1250)\r\ntensor(0.1667)\r\n```\r\n\r\n\r\n### Expected behavior\r\n\r\nPredicting a '0' here should not modify the score, both should give 0.1667\r\n\r\n### Environment\r\n\r\n - PyTorch Version (e.g., 1.0): 1.8.1\r\n - OS (e.g., Linux): Linux & Windows\r\n - How you installed PyTorch (`conda`, `pip`, source): both\r\n - Build command you used (if compiling from source):\r\n - Python version: 3.8\r\n - CUDA/cuDNN version: \r\n - GPU models and configuration:\r\n - Any other relevant information: \r\n\r\n### Additional context\r\n\r\nThe ignore_index is implemented like [this](https://github.com/PyTorchLightning/metrics/blob/master/torchmetrics/functional/classification/iou.py#L39):\r\n```\r\nif ignore_index is not None and 0 <= ignore_index < num_classes:\r\n    scores = torch.cat([\r\n        scores[:ignore_index],\r\n        scores[ignore_index + 1:],\r\n    ])\r\n```\r\nThis is incorrect as it does not provide the right union value. Instead, the confmat should be stripped of its ignore_index column and line.\r\n\r\nThis can be done with this line provided it is put before the computation of the intersection and union:\r\n`confmat = confmat[torch.arange(confmat.size(0)) != ignore_index].T[[ignore_index != torch.arange(confmat.size(0))]].T`\r\n\n", "before_files": [{"content": "# Copyright The PyTorch Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import Optional\n\nimport torch\nfrom torch import Tensor\n\nfrom torchmetrics.functional.classification.confusion_matrix import _confusion_matrix_update\nfrom torchmetrics.utilities.data import get_num_classes\nfrom torchmetrics.utilities.distributed import reduce\n\n\ndef _iou_from_confmat(\n    confmat: Tensor,\n    num_classes: int,\n    ignore_index: Optional[int] = None,\n    absent_score: float = 0.0,\n    reduction: str = 'elementwise_mean',\n) -> Tensor:\n    intersection = torch.diag(confmat)\n    union = confmat.sum(0) + confmat.sum(1) - intersection\n\n    # If this class is absent in both target AND pred (union == 0), then use the absent_score for this class.\n    scores = intersection.float() / union.float()\n    scores[union == 0] = absent_score\n\n    # Remove the ignored class index from the scores.\n    if ignore_index is not None and 0 <= ignore_index < num_classes:\n        scores = torch.cat([\n            scores[:ignore_index],\n            scores[ignore_index + 1:],\n        ])\n    return reduce(scores, reduction=reduction)\n\n\ndef iou(\n    preds: Tensor,\n    target: Tensor,\n    ignore_index: Optional[int] = None,\n    absent_score: float = 0.0,\n    threshold: float = 0.5,\n    num_classes: Optional[int] = None,\n    reduction: str = 'elementwise_mean',\n) -> Tensor:\n    r\"\"\"\n    Computes `Intersection over union, or Jaccard index calculation <https://en.wikipedia.org/wiki/Jaccard_index>`_:\n\n    .. math:: J(A,B) = \\frac{|A\\cap B|}{|A\\cup B|}\n\n    Where: :math:`A` and :math:`B` are both tensors of the same size,\n    containing integer class values. They may be subject to conversion from\n    input data (see description below).\n\n    Note that it is different from box IoU.\n\n    If preds and target are the same shape and preds is a float tensor, we use the ``self.threshold`` argument\n    to convert into integer labels. This is the case for binary and multi-label probabilities.\n\n    If pred has an extra dimension as in the case of multi-class scores we\n    perform an argmax on ``dim=1``.\n\n    Args:\n        preds: tensor containing predictions from model (probabilities, or labels) with shape ``[N, d1, d2, ...]``\n        target: tensor containing ground truth labels with shape ``[N, d1, d2, ...]``\n        ignore_index: optional int specifying a target class to ignore. If given,\n            this class index does not contribute to the returned score, regardless\n            of reduction method. Has no effect if given an int that is not in the\n            range [0, num_classes-1], where num_classes is either given or derived\n            from pred and target. By default, no index is ignored, and all classes are used.\n        absent_score: score to use for an individual class, if no instances of\n            the class index were present in `pred` AND no instances of the class\n            index were present in `target`. For example, if we have 3 classes,\n            [0, 0] for `pred`, and [0, 2] for `target`, then class 1 would be\n            assigned the `absent_score`.\n        threshold:\n            Threshold value for binary or multi-label probabilities. default: 0.5\n        num_classes:\n            Optionally specify the number of classes\n        reduction: a method to reduce metric score over labels.\n\n            - ``'elementwise_mean'``: takes the mean (default)\n            - ``'sum'``: takes the sum\n            - ``'none'``: no reduction will be applied\n\n    Return:\n        IoU score : Tensor containing single value if reduction is\n        'elementwise_mean', or number of classes if reduction is 'none'\n\n    Example:\n        >>> from torchmetrics.functional import iou\n        >>> target = torch.randint(0, 2, (10, 25, 25))\n        >>> pred = torch.tensor(target)\n        >>> pred[2:5, 7:13, 9:15] = 1 - pred[2:5, 7:13, 9:15]\n        >>> iou(pred, target)\n        tensor(0.9660)\n    \"\"\"\n\n    num_classes = get_num_classes(preds=preds, target=target, num_classes=num_classes)\n    confmat = _confusion_matrix_update(preds, target, num_classes, threshold)\n    return _iou_from_confmat(confmat, num_classes, ignore_index, absent_score, reduction)\n", "path": "torchmetrics/functional/classification/iou.py"}], "after_files": [{"content": "# Copyright The PyTorch Lightning team.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import Optional\n\nimport torch\nfrom torch import Tensor\n\nfrom torchmetrics.functional.classification.confusion_matrix import _confusion_matrix_update\nfrom torchmetrics.utilities.data import get_num_classes\nfrom torchmetrics.utilities.distributed import reduce\n\n\ndef _iou_from_confmat(\n    confmat: Tensor,\n    num_classes: int,\n    ignore_index: Optional[int] = None,\n    absent_score: float = 0.0,\n    reduction: str = 'elementwise_mean',\n) -> Tensor:\n\n    # Remove the ignored class index from the scores.\n    if ignore_index is not None and 0 <= ignore_index < num_classes:\n        confmat[ignore_index] = 0.\n\n    intersection = torch.diag(confmat)\n    union = confmat.sum(0) + confmat.sum(1) - intersection\n\n    # If this class is absent in both target AND pred (union == 0), then use the absent_score for this class.\n    scores = intersection.float() / union.float()\n    scores[union == 0] = absent_score\n\n    if ignore_index is not None and 0 <= ignore_index < num_classes:\n        scores = torch.cat([\n            scores[:ignore_index],\n            scores[ignore_index + 1:],\n        ])\n\n    return reduce(scores, reduction=reduction)\n\n\ndef iou(\n    preds: Tensor,\n    target: Tensor,\n    ignore_index: Optional[int] = None,\n    absent_score: float = 0.0,\n    threshold: float = 0.5,\n    num_classes: Optional[int] = None,\n    reduction: str = 'elementwise_mean',\n) -> Tensor:\n    r\"\"\"\n    Computes `Intersection over union, or Jaccard index calculation <https://en.wikipedia.org/wiki/Jaccard_index>`_:\n\n    .. math:: J(A,B) = \\frac{|A\\cap B|}{|A\\cup B|}\n\n    Where: :math:`A` and :math:`B` are both tensors of the same size,\n    containing integer class values. They may be subject to conversion from\n    input data (see description below).\n\n    Note that it is different from box IoU.\n\n    If preds and target are the same shape and preds is a float tensor, we use the ``self.threshold`` argument\n    to convert into integer labels. This is the case for binary and multi-label probabilities.\n\n    If pred has an extra dimension as in the case of multi-class scores we\n    perform an argmax on ``dim=1``.\n\n    Args:\n        preds: tensor containing predictions from model (probabilities, or labels) with shape ``[N, d1, d2, ...]``\n        target: tensor containing ground truth labels with shape ``[N, d1, d2, ...]``\n        ignore_index: optional int specifying a target class to ignore. If given,\n            this class index does not contribute to the returned score, regardless\n            of reduction method. Has no effect if given an int that is not in the\n            range [0, num_classes-1], where num_classes is either given or derived\n            from pred and target. By default, no index is ignored, and all classes are used.\n        absent_score: score to use for an individual class, if no instances of\n            the class index were present in `pred` AND no instances of the class\n            index were present in `target`. For example, if we have 3 classes,\n            [0, 0] for `pred`, and [0, 2] for `target`, then class 1 would be\n            assigned the `absent_score`.\n        threshold:\n            Threshold value for binary or multi-label probabilities. default: 0.5\n        num_classes:\n            Optionally specify the number of classes\n        reduction: a method to reduce metric score over labels.\n\n            - ``'elementwise_mean'``: takes the mean (default)\n            - ``'sum'``: takes the sum\n            - ``'none'``: no reduction will be applied\n\n    Return:\n        IoU score : Tensor containing single value if reduction is\n        'elementwise_mean', or number of classes if reduction is 'none'\n\n    Example:\n        >>> from torchmetrics.functional import iou\n        >>> target = torch.randint(0, 2, (10, 25, 25))\n        >>> pred = torch.tensor(target)\n        >>> pred[2:5, 7:13, 9:15] = 1 - pred[2:5, 7:13, 9:15]\n        >>> iou(pred, target)\n        tensor(0.9660)\n    \"\"\"\n\n    num_classes = get_num_classes(preds=preds, target=target, num_classes=num_classes)\n    confmat = _confusion_matrix_update(preds, target, num_classes, threshold)\n    return _iou_from_confmat(confmat, num_classes, ignore_index, absent_score, reduction)\n", "path": "torchmetrics/functional/classification/iou.py"}]}