prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  [Bug] The Added Loss term for InducingKernel seems flipped in sign?
  # 🐛 Bug
  
  <!-- A clear and concise description of what the bug is. -->
  ```
      def loss(self, *params):
          prior_covar = self.prior_dist.lazy_covariance_matrix
          variational_covar = self.variational_dist.lazy_covariance_matrix
          diag = prior_covar.diag() - variational_covar.diag()
          shape = prior_covar.shape[:-1]
          noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
          return 0.5 * (diag / noise_diag).sum()
  ```
  This is the current code for InducingPointKernelAddedLossTerm.loss
  
  From what I see, this "loss term" is added into the mll that is returned by the `ExactMarginalLogLikelihood` class. This in itself is misleading as the loss is usually the negative of the mll.
  
  Moreover, the variational negative loss used to evaluate inducing points is given below
  
  ![image](https://user-images.githubusercontent.com/4582526/122106510-4c545980-ce1a-11eb-9bbc-681bea511bec.png)
  
  As can be seen, the above is the expression for the pseudo-mll that is maximized when optimizing the inducing points. in this, the component of `InducingPointKernelAddedLossTerm` is negative to the value that is being added into the loss.
  
  This is quite likely a significant bug. Please fix (just invert the sign of `diag` above)
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `gpytorch/mlls/inducing_point_kernel_added_loss_term.py`
  Content:
  ```
  1 #!/usr/bin/env python3
  2 
  3 from .added_loss_term import AddedLossTerm
  4 
  5 
  6 class InducingPointKernelAddedLossTerm(AddedLossTerm):
  7     def __init__(self, variational_dist, prior_dist, likelihood):
  8         self.prior_dist = prior_dist
  9         self.variational_dist = variational_dist
  10         self.likelihood = likelihood
  11 
  12     def loss(self, *params):
  13         prior_covar = self.prior_dist.lazy_covariance_matrix
  14         variational_covar = self.variational_dist.lazy_covariance_matrix
  15         diag = prior_covar.diag() - variational_covar.diag()
  16         shape = prior_covar.shape[:-1]
  17         noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
  18         return 0.5 * (diag / noise_diag).sum()
  19
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
  --- a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
  +++ b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py
  @@ -4,7 +4,7 @@
   
   
   class InducingPointKernelAddedLossTerm(AddedLossTerm):
  -    def __init__(self, variational_dist, prior_dist, likelihood):
  +    def __init__(self, prior_dist, variational_dist, likelihood):
           self.prior_dist = prior_dist
           self.variational_dist = variational_dist
           self.likelihood = likelihood
  @@ -12,7 +12,7 @@
       def loss(self, *params):
           prior_covar = self.prior_dist.lazy_covariance_matrix
           variational_covar = self.variational_dist.lazy_covariance_matrix
  -        diag = prior_covar.diag() - variational_covar.diag()
  +        diag = variational_covar.diag() - prior_covar.diag()
           shape = prior_covar.shape[:-1]
           noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()
           return 0.5 * (diag / noise_diag).sum()
  
verification_info: {"golden_diff": "diff --git a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n--- a/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n+++ b/gpytorch/mlls/inducing_point_kernel_added_loss_term.py\n@@ -4,7 +4,7 @@\n \n \n class InducingPointKernelAddedLossTerm(AddedLossTerm):\n-    def __init__(self, variational_dist, prior_dist, likelihood):\n+    def __init__(self, prior_dist, variational_dist, likelihood):\n         self.prior_dist = prior_dist\n         self.variational_dist = variational_dist\n         self.likelihood = likelihood\n@@ -12,7 +12,7 @@\n     def loss(self, *params):\n         prior_covar = self.prior_dist.lazy_covariance_matrix\n         variational_covar = self.variational_dist.lazy_covariance_matrix\n-        diag = prior_covar.diag() - variational_covar.diag()\n+        diag = variational_covar.diag() - prior_covar.diag()\n         shape = prior_covar.shape[:-1]\n         noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n         return 0.5 * (diag / noise_diag).sum()\n", "issue": "[Bug] The Added Loss term for InducingKernel seems flipped in sign?\n# \ud83d\udc1b Bug\r\n\r\n<!-- A clear and concise description of what the bug is. -->\r\n```\r\n    def loss(self, *params):\r\n        prior_covar = self.prior_dist.lazy_covariance_matrix\r\n        variational_covar = self.variational_dist.lazy_covariance_matrix\r\n        diag = prior_covar.diag() - variational_covar.diag()\r\n        shape = prior_covar.shape[:-1]\r\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\r\n        return 0.5 * (diag / noise_diag).sum()\r\n```\r\nThis is the current code for InducingPointKernelAddedLossTerm.loss\r\n\r\nFrom what I see, this \"loss term\" is added into the mll that is returned by the `ExactMarginalLogLikelihood` class. This in itself is misleading as the loss is usually the negative of the mll.\r\n\r\nMoreover, the variational negative loss used to evaluate inducing points is given below\r\n\r\n![image](https://user-images.githubusercontent.com/4582526/122106510-4c545980-ce1a-11eb-9bbc-681bea511bec.png)\r\n\r\nAs can be seen, the above is the expression for the pseudo-mll that is maximized when optimizing the inducing points. in this, the component of `InducingPointKernelAddedLossTerm` is negative to the value that is being added into the loss.\r\n\r\nThis is quite likely a significant bug. Please fix (just invert the sign of `diag` above)\r\n\n", "before_files": [{"content": "#!/usr/bin/env python3\n\nfrom .added_loss_term import AddedLossTerm\n\n\nclass InducingPointKernelAddedLossTerm(AddedLossTerm):\n    def __init__(self, variational_dist, prior_dist, likelihood):\n        self.prior_dist = prior_dist\n        self.variational_dist = variational_dist\n        self.likelihood = likelihood\n\n    def loss(self, *params):\n        prior_covar = self.prior_dist.lazy_covariance_matrix\n        variational_covar = self.variational_dist.lazy_covariance_matrix\n        diag = prior_covar.diag() - variational_covar.diag()\n        shape = prior_covar.shape[:-1]\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n        return 0.5 * (diag / noise_diag).sum()\n", "path": "gpytorch/mlls/inducing_point_kernel_added_loss_term.py"}], "after_files": [{"content": "#!/usr/bin/env python3\n\nfrom .added_loss_term import AddedLossTerm\n\n\nclass InducingPointKernelAddedLossTerm(AddedLossTerm):\n    def __init__(self, prior_dist, variational_dist, likelihood):\n        self.prior_dist = prior_dist\n        self.variational_dist = variational_dist\n        self.likelihood = likelihood\n\n    def loss(self, *params):\n        prior_covar = self.prior_dist.lazy_covariance_matrix\n        variational_covar = self.variational_dist.lazy_covariance_matrix\n        diag = variational_covar.diag() - prior_covar.diag()\n        shape = prior_covar.shape[:-1]\n        noise_diag = self.likelihood._shaped_noise_covar(shape, *params).diag()\n        return 0.5 * (diag / noise_diag).sum()\n", "path": "gpytorch/mlls/inducing_point_kernel_added_loss_term.py"}]}