We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
ChangePoints kernel breaks for len(X) != len(X2)
<!-- Lines like this are comments and will be invisible -->

# Bug 
There seems to be a bug for models using the Changepoints kernel, whereby given a model 
m =gpflow.models.GPR(data=(X,y), kernel=k, mean_function=None)
with k a Changepoint kernel, (using a combination of any base kernels), the model won't produce a value for predict_y. 

<!-- A clear and concise description of what the bug is. -->

## To reproduce

**Minimal, reproducible example**
<!-- We need to be able to reproduce the bug by simply copy and pasting your code -->
```python
import numpy as np
import gpflow
X = np.linspace(0,100,100).reshape(100,1)
base_k1 = gpflow.kernels.Matern32(lengthscales=0.2)
base_k2 = gpflow.kernels.Matern32(lengthscales=2.0)
k = gpflow.kernels.ChangePoints([base_k1, base_k2], [0.0], steepness=5.0)
k(X)  # works

N = 25  # anything other than N=100 will reproduce the bug
xx = np.linspace(0,50,N).reshape(N,1)
k(X, xx)  # breaks
```

**Stack trace, or error message**
```
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-25-d1dbc7941bae> in <module>
----> 1 k(X, xx)

~/Code/GPflow/gpflow/kernels/base.py in __call__(self, X, X2, full_cov, presliced)
    170 
    171         else:
--> 172             return self.K(X, X2)
    173 
    174     def __add__(self, other):

~/Code/GPflow/gpflow/kernels/changepoints.py in K(self, X, X2)
     83         N = tf.shape(X)[0]
     84         ones = tf.ones((N, N, 1), dtype=X.dtype)
---> 85         starters = tf.concat([ones, starters], axis=2)
     86         stoppers = tf.concat([stoppers, ones], axis=2)
     87 

~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py in wrapper(*args, **kwargs)
    178     """Call target, and fall back on dispatchers if there is a TypeError."""
    179     try:
--> 180       return target(*args, **kwargs)
    181     except (TypeError, ValueError):
    182       # Note: convert_to_eager_tensor currently raises a ValueError, not a

~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py in concat(values, axis, name)
   1515           dtype=dtypes.int32).get_shape().assert_has_rank(0)
   1516       return identity(values[0], name=name)
-> 1517   return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
   1518 
   1519 

~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py in concat_v2(values, axis, name)
   1116         pass  # Add nodes to the TensorFlow graph.
   1117     except _core._NotOkStatusException as e:
-> 1118       _ops.raise_from_not_ok_status(e, name)
   1119   # Add nodes to the TensorFlow graph.
   1120   if not isinstance(values, (list, tuple)):

~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6604   message = e.message + (" name: " + name if name is not None else "")
   6605   # pylint: disable=protected-access
-> 6606   six.raise_from(core._status_to_exception(e.code, message), None)
   6607   # pylint: enable=protected-access
   6608 

~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [100,100,1] vs. shape[1] = [100,25,1] [Op:ConcatV2] name: concat
```

## Expected behavior

Should return the kernel matrix of shape (100, 25).

## System information

* GPflow version: 2.0.1 (reproduced on `develop`)
* GPflow installed from: 'pip install gpflow'
* TensorFlow version: 2.1.0
* Python version  3.7.5
* Operating system Ubuntu Linux (18.04.3)

## Additional context

For me changing these two lines in changepoints.py or kernels did the trick: 

```python
    def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:
        sig_X = self._sigmoids(X)  # N x 1 x Ncp
        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X

        # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go
        # from 1 -> 0, dimensions are N x N x Ncp
        starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))
        stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))

        # prepend `starters` with ones and append ones to `stoppers` since the
        # first kernel has no start and the last kernel has no end
        N = tf.shape(X)[0]
        M = tf.shape(X2)[0] if X2 is not None else N # THIS IS THE FIX
        ones = tf.ones((N, M, 1), dtype=X.dtype) #PREVIOUSLY N WAS IN PLACE OF M HERE
        starters = tf.concat([ones, starters], axis=2)
        stoppers = tf.concat([stoppers, ones], axis=2)
```

However I have not run any kind of tests to make sure this would not break anything else (neither have I added a unit test for the changepoint kernel). It could be that I am using the changepoint kernel incorrectly, in which case it would be very helpful if a short demo of using the kernel in a model could be appended to the changepoint kernel notebook :)


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `gpflow/kernels/changepoints.py`
Content:
```
1 from collections.abc import Iterable
2 from typing import List, Optional, Union
3 
4 import tensorflow as tf
5 
6 from ..base import Parameter
7 from ..utilities import positive
8 from .base import Combination, Kernel
9 
10 
11 class ChangePoints(Combination):
12     r"""
13     The ChangePoints kernel defines a fixed number of change-points along a 1d
14     input space where different kernels govern different parts of the space.
15 
16     The kernel is by multiplication and addition of the base kernels with
17     sigmoid functions (σ). A single change-point kernel is defined as:
18 
19         K₁(x, x') * (1 - σ(x)) * (1 - σ(x')) + K₂(x, x') * σ(x) * σ(x')
20 
21     where K₁ is deactivated around the change-point and K₂ is activated. The
22     single change-point version can be found in \citet{lloyd2014}. Each sigmoid
23     is a logistic function defined as:
24 
25         σ(x) = 1 / (1 + exp{-s(x - x₀)})
26 
27     parameterized by location "x₀" and steepness "s".
28 
29     @incollection{lloyd2014,
30       author = {Lloyd, James Robert et al},
31       title = {Automatic Construction and Natural-language Description of Nonparametric Regression Models},
32       booktitle = {Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence},
33       year = {2014},
34       url = {http://dl.acm.org/citation.cfm?id=2893873.2894066},
35     }
36     """
37 
38     def __init__(
39         self,
40         kernels: List[Kernel],
41         locations: List[float],
42         steepness: Union[float, List[float]] = 1.0,
43         name: Optional[str] = None,
44     ):
45         """
46         :param kernels: list of kernels defining the different regimes
47         :param locations: list of change-point locations in the 1d input space
48         :param steepness: the steepness parameter(s) of the sigmoids, this can be
49             common between them or decoupled
50         """
51         if len(kernels) != len(locations) + 1:
52             raise ValueError(
53                 "Number of kernels ({nk}) must be one more than the number of "
54                 "changepoint locations ({nl})".format(nk=len(kernels), nl=len(locations))
55             )
56 
57         if isinstance(steepness, Iterable) and len(steepness) != len(locations):
58             raise ValueError(
59                 "Dimension of steepness ({ns}) does not match number of changepoint "
60                 "locations ({nl})".format(ns=len(steepness), nl=len(locations))
61             )
62 
63         super().__init__(kernels, name=name)
64 
65         self.locations = Parameter(locations)
66         self.steepness = Parameter(steepness, transform=positive())
67 
68     def _set_kernels(self, kernels: List[Kernel]):
69         # it is not clear how to flatten out nested change-points
70         self.kernels = kernels
71 
72     def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:
73         sig_X = self._sigmoids(X)  # N x 1 x Ncp
74         sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X
75 
76         # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go
77         # from 1 -> 0, dimensions are N x N x Ncp
78         starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))
79         stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))
80 
81         # prepend `starters` with ones and append ones to `stoppers` since the
82         # first kernel has no start and the last kernel has no end
83         N = tf.shape(X)[0]
84         ones = tf.ones((N, N, 1), dtype=X.dtype)
85         starters = tf.concat([ones, starters], axis=2)
86         stoppers = tf.concat([stoppers, ones], axis=2)
87 
88         # now combine with the underlying kernels
89         kernel_stack = tf.stack([k(X, X2) for k in self.kernels], axis=2)
90         return tf.reduce_sum(kernel_stack * starters * stoppers, axis=2)
91 
92     def K_diag(self, X: tf.Tensor) -> tf.Tensor:
93         N = tf.shape(X)[0]
94         sig_X = tf.reshape(self._sigmoids(X), (N, -1))  # N x Ncp
95 
96         ones = tf.ones((N, 1), dtype=X.dtype)
97         starters = tf.concat([ones, sig_X * sig_X], axis=1)  # N x Ncp
98         stoppers = tf.concat([(1 - sig_X) * (1 - sig_X), ones], axis=1)
99 
100         kernel_stack = tf.stack([k(X, full_cov=False) for k in self.kernels], axis=1)
101         return tf.reduce_sum(kernel_stack * starters * stoppers, axis=1)
102 
103     def _sigmoids(self, X: tf.Tensor) -> tf.Tensor:
104         locations = tf.sort(self.locations)  # ensure locations are ordered
105         locations = tf.reshape(locations, (1, 1, -1))
106         steepness = tf.reshape(self.steepness, (1, 1, -1))
107         return tf.sigmoid(steepness * (X[:, :, None] - locations))
108
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/gpflow/kernels/changepoints.py b/gpflow/kernels/changepoints.py
--- a/gpflow/kernels/changepoints.py
+++ b/gpflow/kernels/changepoints.py
@@ -70,18 +70,19 @@
         self.kernels = kernels
 
     def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:
-        sig_X = self._sigmoids(X)  # N x 1 x Ncp
-        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X
+        sig_X = self._sigmoids(X)  # N1 x 1 x Ncp
+        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X  # N2 x 1 x Ncp
 
         # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go
-        # from 1 -> 0, dimensions are N x N x Ncp
+        # from 1 -> 0, dimensions are N1 x N2 x Ncp
         starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))
         stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))
 
         # prepend `starters` with ones and append ones to `stoppers` since the
         # first kernel has no start and the last kernel has no end
-        N = tf.shape(X)[0]
-        ones = tf.ones((N, N, 1), dtype=X.dtype)
+        N1 = tf.shape(X)[0]
+        N2 = tf.shape(X2)[0] if X2 is not None else N1
+        ones = tf.ones((N1, N2, 1), dtype=X.dtype)
         starters = tf.concat([ones, starters], axis=2)
         stoppers = tf.concat([stoppers, ones], axis=2)

{"golden_diff": "diff --git a/gpflow/kernels/changepoints.py b/gpflow/kernels/changepoints.py\n--- a/gpflow/kernels/changepoints.py\n+++ b/gpflow/kernels/changepoints.py\n@@ -70,18 +70,19 @@\n         self.kernels = kernels\n \n     def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:\n-        sig_X = self._sigmoids(X)  # N x 1 x Ncp\n-        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X\n+        sig_X = self._sigmoids(X)  # N1 x 1 x Ncp\n+        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X  # N2 x 1 x Ncp\n \n         # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go\n-        # from 1 -> 0, dimensions are N x N x Ncp\n+        # from 1 -> 0, dimensions are N1 x N2 x Ncp\n         starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))\n         stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))\n \n         # prepend `starters` with ones and append ones to `stoppers` since the\n         # first kernel has no start and the last kernel has no end\n-        N = tf.shape(X)[0]\n-        ones = tf.ones((N, N, 1), dtype=X.dtype)\n+        N1 = tf.shape(X)[0]\n+        N2 = tf.shape(X2)[0] if X2 is not None else N1\n+        ones = tf.ones((N1, N2, 1), dtype=X.dtype)\n         starters = tf.concat([ones, starters], axis=2)\n         stoppers = tf.concat([stoppers, ones], axis=2)\n", "issue": "ChangePoints kernel breaks for len(X) != len(X2)\n<!-- Lines like this are comments and will be invisible -->\r\n\r\n# Bug \r\nThere seems to be a bug for models using the Changepoints kernel, whereby given a model \r\nm =gpflow.models.GPR(data=(X,y), kernel=k, mean_function=None)\r\nwith k a Changepoint kernel, (using a combination of any base kernels), the model won't produce a value for predict_y. \r\n\r\n<!-- A clear and concise description of what the bug is. -->\r\n\r\n## To reproduce\r\n\r\n**Minimal, reproducible example**\r\n<!-- We need to be able to reproduce the bug by simply copy and pasting your code -->\r\n```python\r\nimport numpy as np\r\nimport gpflow\r\nX = np.linspace(0,100,100).reshape(100,1)\r\nbase_k1 = gpflow.kernels.Matern32(lengthscales=0.2)\r\nbase_k2 = gpflow.kernels.Matern32(lengthscales=2.0)\r\nk = gpflow.kernels.ChangePoints([base_k1, base_k2], [0.0], steepness=5.0)\r\nk(X)  # works\r\n\r\nN = 25  # anything other than N=100 will reproduce the bug\r\nxx = np.linspace(0,50,N).reshape(N,1)\r\nk(X, xx)  # breaks\r\n```\r\n\r\n**Stack trace, or error message**\r\n```\r\n---------------------------------------------------------------------------\r\nInvalidArgumentError                      Traceback (most recent call last)\r\n<ipython-input-25-d1dbc7941bae> in <module>\r\n----> 1 k(X, xx)\r\n\r\n~/Code/GPflow/gpflow/kernels/base.py in __call__(self, X, X2, full_cov, presliced)\r\n    170 \r\n    171         else:\r\n--> 172             return self.K(X, X2)\r\n    173 \r\n    174     def __add__(self, other):\r\n\r\n~/Code/GPflow/gpflow/kernels/changepoints.py in K(self, X, X2)\r\n     83         N = tf.shape(X)[0]\r\n     84         ones = tf.ones((N, N, 1), dtype=X.dtype)\r\n---> 85         starters = tf.concat([ones, starters], axis=2)\r\n     86         stoppers = tf.concat([stoppers, ones], axis=2)\r\n     87 \r\n\r\n~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py in wrapper(*args, **kwargs)\r\n    178     \"\"\"Call target, and fall back on dispatchers if there is a TypeError.\"\"\"\r\n    179     try:\r\n--> 180       return target(*args, **kwargs)\r\n    181     except (TypeError, ValueError):\r\n    182       # Note: convert_to_eager_tensor currently raises a ValueError, not a\r\n\r\n~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py in concat(values, axis, name)\r\n   1515           dtype=dtypes.int32).get_shape().assert_has_rank(0)\r\n   1516       return identity(values[0], name=name)\r\n-> 1517   return gen_array_ops.concat_v2(values=values, axis=axis, name=name)\r\n   1518 \r\n   1519 \r\n\r\n~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py in concat_v2(values, axis, name)\r\n   1116         pass  # Add nodes to the TensorFlow graph.\r\n   1117     except _core._NotOkStatusException as e:\r\n-> 1118       _ops.raise_from_not_ok_status(e, name)\r\n   1119   # Add nodes to the TensorFlow graph.\r\n   1120   if not isinstance(values, (list, tuple)):\r\n\r\n~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py in raise_from_not_ok_status(e, name)\r\n   6604   message = e.message + (\" name: \" + name if name is not None else \"\")\r\n   6605   # pylint: disable=protected-access\r\n-> 6606   six.raise_from(core._status_to_exception(e.code, message), None)\r\n   6607   # pylint: enable=protected-access\r\n   6608 \r\n\r\n~/anaconda3/envs/gpflux2/lib/python3.7/site-packages/six.py in raise_from(value, from_value)\r\n\r\nInvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [100,100,1] vs. shape[1] = [100,25,1] [Op:ConcatV2] name: concat\r\n```\r\n\r\n## Expected behavior\r\n\r\nShould return the kernel matrix of shape (100, 25).\r\n\r\n## System information\r\n\r\n* GPflow version: 2.0.1 (reproduced on `develop`)\r\n* GPflow installed from: 'pip install gpflow'\r\n* TensorFlow version: 2.1.0\r\n* Python version  3.7.5\r\n* Operating system Ubuntu Linux (18.04.3)\r\n\r\n## Additional context\r\n\r\nFor me changing these two lines in changepoints.py or kernels did the trick: \r\n\r\n```python\r\n    def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:\r\n        sig_X = self._sigmoids(X)  # N x 1 x Ncp\r\n        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X\r\n\r\n        # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go\r\n        # from 1 -> 0, dimensions are N x N x Ncp\r\n        starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))\r\n        stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))\r\n\r\n        # prepend `starters` with ones and append ones to `stoppers` since the\r\n        # first kernel has no start and the last kernel has no end\r\n        N = tf.shape(X)[0]\r\n        M = tf.shape(X2)[0] if X2 is not None else N # THIS IS THE FIX\r\n        ones = tf.ones((N, M, 1), dtype=X.dtype) #PREVIOUSLY N WAS IN PLACE OF M HERE\r\n        starters = tf.concat([ones, starters], axis=2)\r\n        stoppers = tf.concat([stoppers, ones], axis=2)\r\n```\r\n\r\nHowever I have not run any kind of tests to make sure this would not break anything else (neither have I added a unit test for the changepoint kernel). It could be that I am using the changepoint kernel incorrectly, in which case it would be very helpful if a short demo of using the kernel in a model could be appended to the changepoint kernel notebook :)\r\n\n", "before_files": [{"content": "from collections.abc import Iterable\nfrom typing import List, Optional, Union\n\nimport tensorflow as tf\n\nfrom ..base import Parameter\nfrom ..utilities import positive\nfrom .base import Combination, Kernel\n\n\nclass ChangePoints(Combination):\n    r\"\"\"\n    The ChangePoints kernel defines a fixed number of change-points along a 1d\n    input space where different kernels govern different parts of the space.\n\n    The kernel is by multiplication and addition of the base kernels with\n    sigmoid functions (\u03c3). A single change-point kernel is defined as:\n\n        K\u2081(x, x') * (1 - \u03c3(x)) * (1 - \u03c3(x')) + K\u2082(x, x') * \u03c3(x) * \u03c3(x')\n\n    where K\u2081 is deactivated around the change-point and K\u2082 is activated. The\n    single change-point version can be found in \\citet{lloyd2014}. Each sigmoid\n    is a logistic function defined as:\n\n        \u03c3(x) = 1 / (1 + exp{-s(x - x\u2080)})\n\n    parameterized by location \"x\u2080\" and steepness \"s\".\n\n    @incollection{lloyd2014,\n      author = {Lloyd, James Robert et al},\n      title = {Automatic Construction and Natural-language Description of Nonparametric Regression Models},\n      booktitle = {Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence},\n      year = {2014},\n      url = {http://dl.acm.org/citation.cfm?id=2893873.2894066},\n    }\n    \"\"\"\n\n    def __init__(\n        self,\n        kernels: List[Kernel],\n        locations: List[float],\n        steepness: Union[float, List[float]] = 1.0,\n        name: Optional[str] = None,\n    ):\n        \"\"\"\n        :param kernels: list of kernels defining the different regimes\n        :param locations: list of change-point locations in the 1d input space\n        :param steepness: the steepness parameter(s) of the sigmoids, this can be\n            common between them or decoupled\n        \"\"\"\n        if len(kernels) != len(locations) + 1:\n            raise ValueError(\n                \"Number of kernels ({nk}) must be one more than the number of \"\n                \"changepoint locations ({nl})\".format(nk=len(kernels), nl=len(locations))\n            )\n\n        if isinstance(steepness, Iterable) and len(steepness) != len(locations):\n            raise ValueError(\n                \"Dimension of steepness ({ns}) does not match number of changepoint \"\n                \"locations ({nl})\".format(ns=len(steepness), nl=len(locations))\n            )\n\n        super().__init__(kernels, name=name)\n\n        self.locations = Parameter(locations)\n        self.steepness = Parameter(steepness, transform=positive())\n\n    def _set_kernels(self, kernels: List[Kernel]):\n        # it is not clear how to flatten out nested change-points\n        self.kernels = kernels\n\n    def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:\n        sig_X = self._sigmoids(X)  # N x 1 x Ncp\n        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X\n\n        # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go\n        # from 1 -> 0, dimensions are N x N x Ncp\n        starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))\n        stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))\n\n        # prepend `starters` with ones and append ones to `stoppers` since the\n        # first kernel has no start and the last kernel has no end\n        N = tf.shape(X)[0]\n        ones = tf.ones((N, N, 1), dtype=X.dtype)\n        starters = tf.concat([ones, starters], axis=2)\n        stoppers = tf.concat([stoppers, ones], axis=2)\n\n        # now combine with the underlying kernels\n        kernel_stack = tf.stack([k(X, X2) for k in self.kernels], axis=2)\n        return tf.reduce_sum(kernel_stack * starters * stoppers, axis=2)\n\n    def K_diag(self, X: tf.Tensor) -> tf.Tensor:\n        N = tf.shape(X)[0]\n        sig_X = tf.reshape(self._sigmoids(X), (N, -1))  # N x Ncp\n\n        ones = tf.ones((N, 1), dtype=X.dtype)\n        starters = tf.concat([ones, sig_X * sig_X], axis=1)  # N x Ncp\n        stoppers = tf.concat([(1 - sig_X) * (1 - sig_X), ones], axis=1)\n\n        kernel_stack = tf.stack([k(X, full_cov=False) for k in self.kernels], axis=1)\n        return tf.reduce_sum(kernel_stack * starters * stoppers, axis=1)\n\n    def _sigmoids(self, X: tf.Tensor) -> tf.Tensor:\n        locations = tf.sort(self.locations)  # ensure locations are ordered\n        locations = tf.reshape(locations, (1, 1, -1))\n        steepness = tf.reshape(self.steepness, (1, 1, -1))\n        return tf.sigmoid(steepness * (X[:, :, None] - locations))\n", "path": "gpflow/kernels/changepoints.py"}], "after_files": [{"content": "from collections.abc import Iterable\nfrom typing import List, Optional, Union\n\nimport tensorflow as tf\n\nfrom ..base import Parameter\nfrom ..utilities import positive\nfrom .base import Combination, Kernel\n\n\nclass ChangePoints(Combination):\n    r\"\"\"\n    The ChangePoints kernel defines a fixed number of change-points along a 1d\n    input space where different kernels govern different parts of the space.\n\n    The kernel is by multiplication and addition of the base kernels with\n    sigmoid functions (\u03c3). A single change-point kernel is defined as:\n\n        K\u2081(x, x') * (1 - \u03c3(x)) * (1 - \u03c3(x')) + K\u2082(x, x') * \u03c3(x) * \u03c3(x')\n\n    where K\u2081 is deactivated around the change-point and K\u2082 is activated. The\n    single change-point version can be found in \\citet{lloyd2014}. Each sigmoid\n    is a logistic function defined as:\n\n        \u03c3(x) = 1 / (1 + exp{-s(x - x\u2080)})\n\n    parameterized by location \"x\u2080\" and steepness \"s\".\n\n    @incollection{lloyd2014,\n      author = {Lloyd, James Robert et al},\n      title = {Automatic Construction and Natural-language Description of Nonparametric Regression Models},\n      booktitle = {Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence},\n      year = {2014},\n      url = {http://dl.acm.org/citation.cfm?id=2893873.2894066},\n    }\n    \"\"\"\n\n    def __init__(\n        self,\n        kernels: List[Kernel],\n        locations: List[float],\n        steepness: Union[float, List[float]] = 1.0,\n        name: Optional[str] = None,\n    ):\n        \"\"\"\n        :param kernels: list of kernels defining the different regimes\n        :param locations: list of change-point locations in the 1d input space\n        :param steepness: the steepness parameter(s) of the sigmoids, this can be\n            common between them or decoupled\n        \"\"\"\n        if len(kernels) != len(locations) + 1:\n            raise ValueError(\n                \"Number of kernels ({nk}) must be one more than the number of \"\n                \"changepoint locations ({nl})\".format(nk=len(kernels), nl=len(locations))\n            )\n\n        if isinstance(steepness, Iterable) and len(steepness) != len(locations):\n            raise ValueError(\n                \"Dimension of steepness ({ns}) does not match number of changepoint \"\n                \"locations ({nl})\".format(ns=len(steepness), nl=len(locations))\n            )\n\n        super().__init__(kernels, name=name)\n\n        self.locations = Parameter(locations)\n        self.steepness = Parameter(steepness, transform=positive())\n\n    def _set_kernels(self, kernels: List[Kernel]):\n        # it is not clear how to flatten out nested change-points\n        self.kernels = kernels\n\n    def K(self, X: tf.Tensor, X2: Optional[tf.Tensor] = None) -> tf.Tensor:\n        sig_X = self._sigmoids(X)  # N1 x 1 x Ncp\n        sig_X2 = self._sigmoids(X2) if X2 is not None else sig_X  # N2 x 1 x Ncp\n\n        # `starters` are the sigmoids going from 0 -> 1, whilst `stoppers` go\n        # from 1 -> 0, dimensions are N1 x N2 x Ncp\n        starters = sig_X * tf.transpose(sig_X2, perm=(1, 0, 2))\n        stoppers = (1 - sig_X) * tf.transpose((1 - sig_X2), perm=(1, 0, 2))\n\n        # prepend `starters` with ones and append ones to `stoppers` since the\n        # first kernel has no start and the last kernel has no end\n        N1 = tf.shape(X)[0]\n        N2 = tf.shape(X2)[0] if X2 is not None else N1\n        ones = tf.ones((N1, N2, 1), dtype=X.dtype)\n        starters = tf.concat([ones, starters], axis=2)\n        stoppers = tf.concat([stoppers, ones], axis=2)\n\n        # now combine with the underlying kernels\n        kernel_stack = tf.stack([k(X, X2) for k in self.kernels], axis=2)\n        return tf.reduce_sum(kernel_stack * starters * stoppers, axis=2)\n\n    def K_diag(self, X: tf.Tensor) -> tf.Tensor:\n        N = tf.shape(X)[0]\n        sig_X = tf.reshape(self._sigmoids(X), (N, -1))  # N x Ncp\n\n        ones = tf.ones((N, 1), dtype=X.dtype)\n        starters = tf.concat([ones, sig_X * sig_X], axis=1)  # N x Ncp\n        stoppers = tf.concat([(1 - sig_X) * (1 - sig_X), ones], axis=1)\n\n        kernel_stack = tf.stack([k(X, full_cov=False) for k in self.kernels], axis=1)\n        return tf.reduce_sum(kernel_stack * starters * stoppers, axis=1)\n\n    def _sigmoids(self, X: tf.Tensor) -> tf.Tensor:\n        locations = tf.sort(self.locations)  # ensure locations are ordered\n        locations = tf.reshape(locations, (1, 1, -1))\n        steepness = tf.reshape(self.steepness, (1, 1, -1))\n        return tf.sigmoid(steepness * (X[:, :, None] - locations))\n", "path": "gpflow/kernels/changepoints.py"}]}