We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[Bug] SumBatchLazyTensor size is inconsistent with indices
# 🐛 Bug

I want to do a KroneckerProductLazyTensor on a batch of lazyTensor `x` times, then SumBatchLazyTensor and then get a specific row and finally evaluate. The code works if I first do an evaluation on the `sum_a` then retrieve the row (which is inefficient) but gives `size is inconsistent with indices` error if I retrieve the row first and then wants to evaluate.

Interestingly, If I use the same number for the dimension -1 and -2, there would be no error then.

## To reproduce

** Code snippet to reproduce **
```python
import gpytorch
x = 3
a = torch.rand((x, 5, 2, 3))
lazy_a = gpytorch.lazy.NonLazyTensor(a)
assert lazy_a.shape == torch.Size([3, 5, 2, 3])
prod_a = gpytorch.lazy.KroneckerProductLazyTensor(*lazy_a)
assert prod_a.shape == torch.Size([5, 8, 27])
sum_a = gpytorch.lazy.SumBatchLazyTensor(prod_a)
assert sum_a.shape == torch.Size([8, 27])
assert sum_a.evaluate()[0].shape == torch.Size([27])
assert sum_a[0].evaluate().shape == torch.Size([27]) # gives error in here
```

** Stack trace/error message **
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-251-7cce10ce99d3> in <module>()
      8 assert sum_a.shape == torch.Size([8, 27])
      9 assert sum_a.evaluate()[0].shape == torch.Size([27])
---> 10 assert sum_a[0].evaluate().shape == torch.Size([27])
     11 

9 frames
/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in __getitem__(self, index)
   1703         # with the appropriate shape
   1704         if (squeeze_row or squeeze_col or row_col_are_absorbed):
-> 1705             res = delazify(res)
   1706         if squeeze_row:
   1707             res = res.squeeze(-2)

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in delazify(obj)
   1753         return obj
   1754     elif isinstance(obj, LazyTensor):
-> 1755         return obj.evaluate()
   1756     else:
   1757         raise TypeError("object of class {} cannot be made into a Tensor".format(obj.__class__.__name__))

/usr/local/lib/python3.6/dist-packages/gpytorch/utils/memoize.py in g(self, *args, **kwargs)
     32         cache_name = name if name is not None else method
     33         if not is_in_cache(self, cache_name):
---> 34             add_to_cache(self, cache_name, method(self, *args, **kwargs))
     35         return get_from_cache(self, cache_name)
     36 

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in evaluate(self)
    858             eye = torch.eye(num_rows, dtype=self.dtype, device=self.device)
    859             eye = eye.expand(*self.batch_shape, num_rows, num_rows)
--> 860             res = self.transpose(-1, -2).matmul(eye).transpose(-1, -2).contiguous()
    861         else:
    862             eye = torch.eye(num_cols, dtype=self.dtype, device=self.device)

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in matmul(self, other)
   1093 
   1094         func = Matmul()
-> 1095         return func.apply(self.representation_tree(), other, *self.representation())
   1096 
   1097     @property

/usr/local/lib/python3.6/dist-packages/gpytorch/functions/_matmul.py in forward(ctx, representation_tree, rhs, *matrix_args)
     18 
     19         lazy_tsr = ctx.representation_tree(*matrix_args)
---> 20         res = lazy_tsr._matmul(rhs)
     21 
     22         to_save = [orig_rhs] + list(matrix_args)

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/block_lazy_tensor.py in _matmul(self, rhs)
     64 
     65         rhs = self._add_batch_dim(rhs)
---> 66         res = self.base_lazy_tensor._matmul(rhs)
     67         res = self._remove_batch_dim(res)
     68 

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/interpolated_lazy_tensor.py in _matmul(self, rhs)
    157     def _matmul(self, rhs):
    158         # Get sparse tensor representations of left/right interp matrices
--> 159         left_interp_t = self._sparse_left_interp_t(self.left_interp_indices, self.left_interp_values)
    160         right_interp_t = self._sparse_right_interp_t(self.right_interp_indices, self.right_interp_values)
    161 

/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/interpolated_lazy_tensor.py in _sparse_left_interp_t(self, left_interp_indices_tensor, left_interp_values_tensor)
    309 
    310         left_interp_t = sparse.make_sparse_from_indices_and_values(
--> 311             left_interp_indices_tensor, left_interp_values_tensor, self.base_lazy_tensor.size()[-1]
    312         )
    313         self._left_interp_indices_memo = left_interp_indices_tensor

/usr/local/lib/python3.6/dist-packages/gpytorch/utils/sparse.py in make_sparse_from_indices_and_values(interp_indices, interp_values, num_rows)
     59     else:
     60         cls = getattr(torch.sparse, type_name)
---> 61     res = cls(index_tensor, value_tensor, interp_size)
     62 
     63     # Wrap things as a variable, if necessary

RuntimeError: size is inconsistent with indices: for dim 1, size is 8 but found index 26
```

## Expected Behavior

Expected to pass the tests.

## System information

**Please complete the following information:**
- GPyTorch Version 0.3.5
- PyTorch Version 1.2.0
- Ubuntu 18.04.3 LTS



--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `gpytorch/lazy/sum_batch_lazy_tensor.py`
Content:
```
1 #!/usr/bin/env python3
2 
3 import torch
4 from .block_lazy_tensor import BlockLazyTensor
5 from ..utils.broadcasting import _pad_with_singletons
6 from ..utils.getitem import _noop_index
7 
8 
9 class SumBatchLazyTensor(BlockLazyTensor):
10     """
11     Represents a lazy tensor that is actually the sum of several lazy tensors blocks.
12     The :attr:`block_dim` attribute specifies which dimension of the base LazyTensor
13     specifies the blocks.
14     For example, (with `block_dim=-3` a `k x n x n` tensor represents `k` `n x n` blocks (a `n x n` matrix).
15     A `b x k x n x n` tensor represents `k` `b x n x n` blocks (a `b x n x n` batch matrix).
16 
17     Args:
18         :attr:`base_lazy_tensor` (LazyTensor):
19             A `k x n x n` LazyTensor, or a `b x k x n x n` LazyTensor.
20         :attr:`block_dim` (int):
21             The dimension that specifies the blocks.
22     """
23     def _add_batch_dim(self, other):
24         shape = list(other.shape)
25         expand_shape = list(other.shape)
26         shape.insert(-2, 1)
27         expand_shape.insert(-2, self.base_lazy_tensor.size(-3))
28         other = other.reshape(*shape).expand(*expand_shape)
29         return other
30 
31     def _get_indices(self, row_index, col_index, *batch_indices):
32         # Create an extra index for the summed dimension
33         sum_index = torch.arange(0, self.base_lazy_tensor.size(-3), device=self.device)
34         sum_index = _pad_with_singletons(sum_index, row_index.dim(), 0)
35         row_index = row_index.unsqueeze(-1)
36         col_index = col_index.unsqueeze(-1)
37         batch_indices = [index.unsqueeze(-1) for index in batch_indices]
38 
39         res = self.base_lazy_tensor._get_indices(row_index, col_index, *batch_indices, sum_index)
40         return res.sum(-1)
41 
42     def _getitem(self, row_index, col_index, *batch_indices):
43         res = self.base_lazy_tensor._getitem(row_index, col_index, *batch_indices, _noop_index)
44         return self.__class__(res, **self._kwargs)
45 
46     def _remove_batch_dim(self, other):
47         return other.sum(-3)
48 
49     def _size(self):
50         shape = list(self.base_lazy_tensor.shape)
51         del shape[-3]
52         return torch.Size(shape)
53 
54     def diag(self):
55         diag = self.base_lazy_tensor.diag().sum(-2)
56         return diag
57
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/gpytorch/lazy/sum_batch_lazy_tensor.py b/gpytorch/lazy/sum_batch_lazy_tensor.py
--- a/gpytorch/lazy/sum_batch_lazy_tensor.py
+++ b/gpytorch/lazy/sum_batch_lazy_tensor.py
@@ -54,3 +54,6 @@
     def diag(self):
         diag = self.base_lazy_tensor.diag().sum(-2)
         return diag
+
+    def evaluate(self):
+        return self.base_lazy_tensor.evaluate().sum(dim=-3)  # BlockLazyTensors always use dim3 for the block_dim

{"golden_diff": "diff --git a/gpytorch/lazy/sum_batch_lazy_tensor.py b/gpytorch/lazy/sum_batch_lazy_tensor.py\n--- a/gpytorch/lazy/sum_batch_lazy_tensor.py\n+++ b/gpytorch/lazy/sum_batch_lazy_tensor.py\n@@ -54,3 +54,6 @@\n     def diag(self):\n         diag = self.base_lazy_tensor.diag().sum(-2)\n         return diag\n+\n+    def evaluate(self):\n+        return self.base_lazy_tensor.evaluate().sum(dim=-3)  # BlockLazyTensors always use dim3 for the block_dim\n", "issue": "[Bug] SumBatchLazyTensor size is inconsistent with indices\n# \ud83d\udc1b Bug\r\n\r\nI want to do a KroneckerProductLazyTensor on a batch of lazyTensor `x` times, then SumBatchLazyTensor and then get a specific row and finally evaluate. The code works if I first do an evaluation on the `sum_a` then retrieve the row (which is inefficient) but gives `size is inconsistent with indices` error if I retrieve the row first and then wants to evaluate.\r\n\r\nInterestingly, If I use the same number for the dimension -1 and -2, there would be no error then.\r\n\r\n## To reproduce\r\n\r\n** Code snippet to reproduce **\r\n```python\r\nimport gpytorch\r\nx = 3\r\na = torch.rand((x, 5, 2, 3))\r\nlazy_a = gpytorch.lazy.NonLazyTensor(a)\r\nassert lazy_a.shape == torch.Size([3, 5, 2, 3])\r\nprod_a = gpytorch.lazy.KroneckerProductLazyTensor(*lazy_a)\r\nassert prod_a.shape == torch.Size([5, 8, 27])\r\nsum_a = gpytorch.lazy.SumBatchLazyTensor(prod_a)\r\nassert sum_a.shape == torch.Size([8, 27])\r\nassert sum_a.evaluate()[0].shape == torch.Size([27])\r\nassert sum_a[0].evaluate().shape == torch.Size([27]) # gives error in here\r\n```\r\n\r\n** Stack trace/error message **\r\n```\r\n---------------------------------------------------------------------------\r\nRuntimeError                              Traceback (most recent call last)\r\n<ipython-input-251-7cce10ce99d3> in <module>()\r\n      8 assert sum_a.shape == torch.Size([8, 27])\r\n      9 assert sum_a.evaluate()[0].shape == torch.Size([27])\r\n---> 10 assert sum_a[0].evaluate().shape == torch.Size([27])\r\n     11 \r\n\r\n9 frames\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in __getitem__(self, index)\r\n   1703         # with the appropriate shape\r\n   1704         if (squeeze_row or squeeze_col or row_col_are_absorbed):\r\n-> 1705             res = delazify(res)\r\n   1706         if squeeze_row:\r\n   1707             res = res.squeeze(-2)\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in delazify(obj)\r\n   1753         return obj\r\n   1754     elif isinstance(obj, LazyTensor):\r\n-> 1755         return obj.evaluate()\r\n   1756     else:\r\n   1757         raise TypeError(\"object of class {} cannot be made into a Tensor\".format(obj.__class__.__name__))\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/utils/memoize.py in g(self, *args, **kwargs)\r\n     32         cache_name = name if name is not None else method\r\n     33         if not is_in_cache(self, cache_name):\r\n---> 34             add_to_cache(self, cache_name, method(self, *args, **kwargs))\r\n     35         return get_from_cache(self, cache_name)\r\n     36 \r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in evaluate(self)\r\n    858             eye = torch.eye(num_rows, dtype=self.dtype, device=self.device)\r\n    859             eye = eye.expand(*self.batch_shape, num_rows, num_rows)\r\n--> 860             res = self.transpose(-1, -2).matmul(eye).transpose(-1, -2).contiguous()\r\n    861         else:\r\n    862             eye = torch.eye(num_cols, dtype=self.dtype, device=self.device)\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/lazy_tensor.py in matmul(self, other)\r\n   1093 \r\n   1094         func = Matmul()\r\n-> 1095         return func.apply(self.representation_tree(), other, *self.representation())\r\n   1096 \r\n   1097     @property\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/functions/_matmul.py in forward(ctx, representation_tree, rhs, *matrix_args)\r\n     18 \r\n     19         lazy_tsr = ctx.representation_tree(*matrix_args)\r\n---> 20         res = lazy_tsr._matmul(rhs)\r\n     21 \r\n     22         to_save = [orig_rhs] + list(matrix_args)\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/block_lazy_tensor.py in _matmul(self, rhs)\r\n     64 \r\n     65         rhs = self._add_batch_dim(rhs)\r\n---> 66         res = self.base_lazy_tensor._matmul(rhs)\r\n     67         res = self._remove_batch_dim(res)\r\n     68 \r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/interpolated_lazy_tensor.py in _matmul(self, rhs)\r\n    157     def _matmul(self, rhs):\r\n    158         # Get sparse tensor representations of left/right interp matrices\r\n--> 159         left_interp_t = self._sparse_left_interp_t(self.left_interp_indices, self.left_interp_values)\r\n    160         right_interp_t = self._sparse_right_interp_t(self.right_interp_indices, self.right_interp_values)\r\n    161 \r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/lazy/interpolated_lazy_tensor.py in _sparse_left_interp_t(self, left_interp_indices_tensor, left_interp_values_tensor)\r\n    309 \r\n    310         left_interp_t = sparse.make_sparse_from_indices_and_values(\r\n--> 311             left_interp_indices_tensor, left_interp_values_tensor, self.base_lazy_tensor.size()[-1]\r\n    312         )\r\n    313         self._left_interp_indices_memo = left_interp_indices_tensor\r\n\r\n/usr/local/lib/python3.6/dist-packages/gpytorch/utils/sparse.py in make_sparse_from_indices_and_values(interp_indices, interp_values, num_rows)\r\n     59     else:\r\n     60         cls = getattr(torch.sparse, type_name)\r\n---> 61     res = cls(index_tensor, value_tensor, interp_size)\r\n     62 \r\n     63     # Wrap things as a variable, if necessary\r\n\r\nRuntimeError: size is inconsistent with indices: for dim 1, size is 8 but found index 26\r\n```\r\n\r\n## Expected Behavior\r\n\r\nExpected to pass the tests.\r\n\r\n## System information\r\n\r\n**Please complete the following information:**\r\n- GPyTorch Version 0.3.5\r\n- PyTorch Version 1.2.0\r\n- Ubuntu 18.04.3 LTS\r\n\r\n\n", "before_files": [{"content": "#!/usr/bin/env python3\n\nimport torch\nfrom .block_lazy_tensor import BlockLazyTensor\nfrom ..utils.broadcasting import _pad_with_singletons\nfrom ..utils.getitem import _noop_index\n\n\nclass SumBatchLazyTensor(BlockLazyTensor):\n    \"\"\"\n    Represents a lazy tensor that is actually the sum of several lazy tensors blocks.\n    The :attr:`block_dim` attribute specifies which dimension of the base LazyTensor\n    specifies the blocks.\n    For example, (with `block_dim=-3` a `k x n x n` tensor represents `k` `n x n` blocks (a `n x n` matrix).\n    A `b x k x n x n` tensor represents `k` `b x n x n` blocks (a `b x n x n` batch matrix).\n\n    Args:\n        :attr:`base_lazy_tensor` (LazyTensor):\n            A `k x n x n` LazyTensor, or a `b x k x n x n` LazyTensor.\n        :attr:`block_dim` (int):\n            The dimension that specifies the blocks.\n    \"\"\"\n    def _add_batch_dim(self, other):\n        shape = list(other.shape)\n        expand_shape = list(other.shape)\n        shape.insert(-2, 1)\n        expand_shape.insert(-2, self.base_lazy_tensor.size(-3))\n        other = other.reshape(*shape).expand(*expand_shape)\n        return other\n\n    def _get_indices(self, row_index, col_index, *batch_indices):\n        # Create an extra index for the summed dimension\n        sum_index = torch.arange(0, self.base_lazy_tensor.size(-3), device=self.device)\n        sum_index = _pad_with_singletons(sum_index, row_index.dim(), 0)\n        row_index = row_index.unsqueeze(-1)\n        col_index = col_index.unsqueeze(-1)\n        batch_indices = [index.unsqueeze(-1) for index in batch_indices]\n\n        res = self.base_lazy_tensor._get_indices(row_index, col_index, *batch_indices, sum_index)\n        return res.sum(-1)\n\n    def _getitem(self, row_index, col_index, *batch_indices):\n        res = self.base_lazy_tensor._getitem(row_index, col_index, *batch_indices, _noop_index)\n        return self.__class__(res, **self._kwargs)\n\n    def _remove_batch_dim(self, other):\n        return other.sum(-3)\n\n    def _size(self):\n        shape = list(self.base_lazy_tensor.shape)\n        del shape[-3]\n        return torch.Size(shape)\n\n    def diag(self):\n        diag = self.base_lazy_tensor.diag().sum(-2)\n        return diag\n", "path": "gpytorch/lazy/sum_batch_lazy_tensor.py"}], "after_files": [{"content": "#!/usr/bin/env python3\n\nimport torch\nfrom .block_lazy_tensor import BlockLazyTensor\nfrom ..utils.broadcasting import _pad_with_singletons\nfrom ..utils.getitem import _noop_index\n\n\nclass SumBatchLazyTensor(BlockLazyTensor):\n    \"\"\"\n    Represents a lazy tensor that is actually the sum of several lazy tensors blocks.\n    The :attr:`block_dim` attribute specifies which dimension of the base LazyTensor\n    specifies the blocks.\n    For example, (with `block_dim=-3` a `k x n x n` tensor represents `k` `n x n` blocks (a `n x n` matrix).\n    A `b x k x n x n` tensor represents `k` `b x n x n` blocks (a `b x n x n` batch matrix).\n\n    Args:\n        :attr:`base_lazy_tensor` (LazyTensor):\n            A `k x n x n` LazyTensor, or a `b x k x n x n` LazyTensor.\n        :attr:`block_dim` (int):\n            The dimension that specifies the blocks.\n    \"\"\"\n    def _add_batch_dim(self, other):\n        shape = list(other.shape)\n        expand_shape = list(other.shape)\n        shape.insert(-2, 1)\n        expand_shape.insert(-2, self.base_lazy_tensor.size(-3))\n        other = other.reshape(*shape).expand(*expand_shape)\n        return other\n\n    def _get_indices(self, row_index, col_index, *batch_indices):\n        # Create an extra index for the summed dimension\n        sum_index = torch.arange(0, self.base_lazy_tensor.size(-3), device=self.device)\n        sum_index = _pad_with_singletons(sum_index, row_index.dim(), 0)\n        row_index = row_index.unsqueeze(-1)\n        col_index = col_index.unsqueeze(-1)\n        batch_indices = [index.unsqueeze(-1) for index in batch_indices]\n\n        res = self.base_lazy_tensor._get_indices(row_index, col_index, *batch_indices, sum_index)\n        return res.sum(-1)\n\n    def _getitem(self, row_index, col_index, *batch_indices):\n        res = self.base_lazy_tensor._getitem(row_index, col_index, *batch_indices, _noop_index)\n        return self.__class__(res, **self._kwargs)\n\n    def _remove_batch_dim(self, other):\n        return other.sum(-3)\n\n    def _size(self):\n        shape = list(self.base_lazy_tensor.shape)\n        del shape[-3]\n        return torch.Size(shape)\n\n    def diag(self):\n        diag = self.base_lazy_tensor.diag().sum(-2)\n        return diag\n\n    def evaluate(self):\n        return self.base_lazy_tensor.evaluate().sum(dim=-3)  # BlockLazyTensors always use dim3 for the block_dim\n", "path": "gpytorch/lazy/sum_batch_lazy_tensor.py"}]}