We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[Bug] The Added Loss term for InducingKernel seems flipped in sign?
# 🐛 Bug

<!-- A clear and concise description of what the bug is. -->
```
    def loss(self, *params):
        prior_covar = self.prior_dist.lazy_covariance_matrix
        variational_covar = self.variational_dist.lazy_covariance_matrix
        diag = prior_covar.diag() - variational_covar.diag()
        shape = prior_covar.shape[:-1]
        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
        return 0.5 * (diag / noise_diag).sum()
```
This is the current code for InducingPointKernelAddedLossTerm.loss

From what I see, this "loss term" is added into the mll that is returned by the `ExactMarginalLogLikelihood` class. This in itself is misleading as the loss is usually the negative of the mll.

Moreover, the variational negative loss used to evaluate inducing points is given below

![image](https://user-images.githubusercontent.com/4582526/122106510-4c545980-ce1a-11eb-9bbc-681bea511bec.png)

As can be seen, the above is the expression for the pseudo-mll that is maximized when optimizing the inducing points. in this, the component of `InducingPointKernelAddedLossTerm` is negative to the value that is being added into the loss.

This is quite likely a significant bug. Please fix (just invert the sign of `diag` above)


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `gpytorch/mlls/inducing_point_kernel_added_loss_term.py`
Content:
```
1 #!/usr/bin/env python3
2 
3 from .added_loss_term import AddedLossTerm
4 
5 
6 class InducingPointKernelAddedLossTerm(AddedLossTerm):
7     def __init__(self, variational_dist, prior_dist, likelihood):
8         self.prior_dist = prior_dist
9         self.variational_dist = variational_dist
10         self.likelihood = likelihood
11 
12     def loss(self, *params):
13         prior_covar = self.prior_dist.lazy_covariance_matrix
14         variational_covar = self.variational_dist.lazy_covariance_matrix
15         diag = prior_covar.diag() - variational_covar.diag()
16         shape = prior_covar.shape[:-1]
17         noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
18         return 0.5 * (diag / noise_diag).sum()
19
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
--- a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
+++ b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
@@ -4,7 +4,7 @@
 
 
 class InducingPointKernelAddedLossTerm(AddedLossTerm):
-    def __init__(self, variational_dist, prior_dist, likelihood):
+    def __init__(self, prior_dist, variational_dist, likelihood):
         self.prior_dist = prior_dist
         self.variational_dist = variational_dist
         self.likelihood = likelihood
@@ -12,7 +12,7 @@
     def loss(self, *params):
         prior_covar = self.prior_dist.lazy_covariance_matrix
         variational_covar = self.variational_dist.lazy_covariance_matrix
-        diag = prior_covar.diag() - variational_covar.diag()
+        diag = variational_covar.diag() - prior_covar.diag()
         shape = prior_covar.shape[:-1]
         noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
         return 0.5 * (diag / noise_diag).sum()

{"golden_diff": "diff --git a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n--- a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n+++ b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n@@ -4,7 +4,7 @@\n \n \n class InducingPointKernelAddedLossTerm(AddedLossTerm):\n-    def __init__(self, variational_dist, prior_dist, likelihood):\n+    def __init__(self, prior_dist, variational_dist, likelihood):\n         self.prior_dist = prior_dist\n         self.variational_dist = variational_dist\n         self.likelihood = likelihood\n@@ -12,7 +12,7 @@\n     def loss(self, *params):\n         prior_covar = self.prior_dist.lazy_covariance_matrix\n         variational_covar = self.variational_dist.lazy_covariance_matrix\n-        diag = prior_covar.diag() - variational_covar.diag()\n+        diag = variational_covar.diag() - prior_covar.diag()\n         shape = prior_covar.shape[:-1]\n         noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n         return 0.5 * (diag / noise_diag).sum()\n", "issue": "[Bug] The Added Loss term for InducingKernel seems flipped in sign?\n# \ud83d\udc1b Bug\r\n\r\n<!-- A clear and concise description of what the bug is. -->\r\n```\r\n    def loss(self, *params):\r\n        prior_covar = self.prior_dist.lazy_covariance_matrix\r\n        variational_covar = self.variational_dist.lazy_covariance_matrix\r\n        diag = prior_covar.diag() - variational_covar.diag()\r\n        shape = prior_covar.shape[:-1]\r\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\r\n        return 0.5 * (diag / noise_diag).sum()\r\n```\r\nThis is the current code for InducingPointKernelAddedLossTerm.loss\r\n\r\nFrom what I see, this \"loss term\" is added into the mll that is returned by the `ExactMarginalLogLikelihood` class. This in itself is misleading as the loss is usually the negative of the mll.\r\n\r\nMoreover, the variational negative loss used to evaluate inducing points is given below\r\n\r\n![image](https://user-images.githubusercontent.com/4582526/122106510-4c545980-ce1a-11eb-9bbc-681bea511bec.png)\r\n\r\nAs can be seen, the above is the expression for the pseudo-mll that is maximized when optimizing the inducing points. in this, the component of `InducingPointKernelAddedLossTerm` is negative to the value that is being added into the loss.\r\n\r\nThis is quite likely a significant bug. Please fix (just invert the sign of `diag` above)\r\n\n", "before_files": [{"content": "#!/usr/bin/env python3\n\nfrom .added_loss_term import AddedLossTerm\n\n\nclass InducingPointKernelAddedLossTerm(AddedLossTerm):\n    def __init__(self, variational_dist, prior_dist, likelihood):\n        self.prior_dist = prior_dist\n        self.variational_dist = variational_dist\n        self.likelihood = likelihood\n\n    def loss(self, *params):\n        prior_covar = self.prior_dist.lazy_covariance_matrix\n        variational_covar = self.variational_dist.lazy_covariance_matrix\n        diag = prior_covar.diag() - variational_covar.diag()\n        shape = prior_covar.shape[:-1]\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n        return 0.5 * (diag / noise_diag).sum()\n", "path": "gpytorch/mlls/inducing_point_kernel_added_loss_term.py"}], "after_files": [{"content": "#!/usr/bin/env python3\n\nfrom .added_loss_term import AddedLossTerm\n\n\nclass InducingPointKernelAddedLossTerm(AddedLossTerm):\n    def __init__(self, prior_dist, variational_dist, likelihood):\n        self.prior_dist = prior_dist\n        self.variational_dist = variational_dist\n        self.likelihood = likelihood\n\n    def loss(self, *params):\n        prior_covar = self.prior_dist.lazy_covariance_matrix\n        variational_covar = self.variational_dist.lazy_covariance_matrix\n        diag = variational_covar.diag() - prior_covar.diag()\n        shape = prior_covar.shape[:-1]\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n        return 0.5 * (diag / noise_diag).sum()\n", "path": "gpytorch/mlls/inducing_point_kernel_added_loss_term.py"}]}