Bayesian Perceptron: Why to marginalize over neuron's output instead of it's weights?
We want a distribution over $w$, don't we?

Yes. You want to obtain a distribution over the parameters, which models the uncertainty about the parameters. This distribution over the parameters can induce a probability distribution over the possible functions consistent with your data.

Why is $a$ integrated out here and not $w$?

This is just the definition of marginalization.
For simplicity, consider 2 discrete random variables $X$ and $Y$, which can take values $x_i$ and $y_i$, respectively, and for which you can have (at least conceptually) a joint distribution $p_{(X, Y)}(x, y)$. Then the marginal probability that $X$ takes the value $x_i$ independently of any outcome of the r.v. $Y$ (and this is the intuition behind the marginal distribution!) is given by
$${\displaystyle p_{X}(x_{i})=\sum _{j}p(x_{i},y_{j})}, \tag{1}\label{1}$$
So, in the case of equation 14 of the paper, there's the assumption that you have $p(w, a)$. Then $p(w)$ is a marginal distribution over $w$ (the parameters or weights of the neural network), independently of the values of the activations. Of course, $w$ and $a$ can be dependent or correlated, and that's why you also have $p(w, a)$, i.e. the probability distribution that describes how these two r.v.s vary together, but what you want to know is the probability distribution over $w$ independently of $a$, so you &quot;marginalize out&quot; the possible values of $a$ (not $w$).
Remember that $p(x, y) = p(x \mid y) p(y) = p(y \mid x) p(x)$ (this is known as the chain rule of probability for two r.v.s, which leads to the Bayes' theorem), so you could write equation (\ref{1}) as follows
$${\displaystyle p_{X}(x_{i})=\sum _{j}p(x_{i},y_{j})} = \sum _{j}p(x_{i} \mid y_{j}) p(y_j), \tag{2}\label{2}$$
In the case of real-valued r.v.s, you will have an integral rather than a summation, but the idea is roughly the same.

But without marginalization over $w$ there is still uncertainty inside $w$.

What we want to know is $p(w)$, i.e. a distribution over $w$, which models the uncertainty about the possible values that $w$ can take. So, once you know $p(w)$, depending on its shape, we will have more or less uncertainty about the possible values of $w$. For example, let's say $p(w)$ is a Gaussian distribution. Then, depending on the variance, you will have more or less uncertainty about the possible values of $w$.
Your doubt seems to be that you thought that marginalization is a way to not have uncertainty, but that's not true. So, the goal in Bayesian machine/deep learning is not to be certain about the values of the parameters, but to model the uncertainty about the values of the parameters.