Is there anything that ensures that convolutional filters end up different from one another? I found this question very interesting, and this is a follow up on it.
Presumably, we'd want all the filters to converge towards some complementary set, where each filter fills as large a niche as possible (in terms of extracting useful information from the previous layer), without overlapping with another filter.
A quick thought experiment tells me (please correct me if I'm wrong) that if two filters are identical down to maximum precision, then without adding in any other form of stochastic differentiation between them, their weights will be updated in the same way at each step of gradient descent during training. Thus, it would be a very bad idea to initialise all filters in the same way prior to training, as they would all be updated in exactly the same way (see footnote 1).
On the other hand, a quick thought experiment isn't enough to tell me what would happen to two filters that are almost identical, as we continue to train the network. Is there some mechanism causing them to then diverge away from one another, thereby filling their own &quot;complementary niches&quot; in the layer? My intuition tells me that there must be, otherwise using many filters just wouldn't work. But during back-propagation, each filter is downstream, and so they don't have any way of communicating with one another. At the risk of anthropomorphising the network, I might ask &quot;How do the two filters collude with one another to benefit the network as a whole?&quot;

Footnotes:

Why do I think this? Because the expession for the partial derivative of the $k$th filter weights with respect to the cost $\partial W^k/\partial C$ will be identical for all $k$. From the perspective of back-propagation, all paths through the filters look exactly the same.
Yes, your thought experiment is correct, and the concept is known as breaking the symmetry. This is why biases can be initialized to $0$ (bias initialization doesn't matter), but weights should be randomly initialized to different numbers -- to break the symmetry. Otherwise, if not, the network will function as if it has $n-1$ filters (or however many filters that are unique) instead of the full $n$ filters.
As for your main question, if two filters are initialized to very similar values, they may branch out as long as that is what minimizes the training loss. There is no collusion or coordination going on; each filter updates completely independently. You can even freeze all the other filters and only perform gradient descent on one filter at a time. Each filter just follows the direction of their gradient to minimize the training loss.
Consider the backprop equations as defined by this online book:

The gradient of the current layer's weights depends on

The future layers' weights, errors, and activation function's derivatives
The current layer's activation function's derivative, and
The previous layer's outputs.

Each weight in the layer (i.e. each filter in the layer) looks at different parts of these three components (indexed by $j$ and $k$ in equation $BP4$). It is this different perspective that allows them to update their gradients in different directions, even if their initial weights are very similar to each other. Note that it is possible that they end up with the same gradient, but it is very unlikely.