Does the Bayesian MAP give a probability distribution over unseen data?
Bayesian MAP gives a point estimate $\mathbf{w}_{MAP}$ for model parameter $\mathbf{w}$, which implies
$p(\mathbf{w}|\mathbf{t},\cdots) = \left\{\begin{matrix}
1 &amp; \mathbf{w}=\mathbf{w}_{MAP}\\ 
0 &amp;\text{o.w.}
\end{matrix}\right.$
However, distribution $p(t_*|x_*, \mathbf{w}_{MAP}, \sigma^2)$ is the probabilistic model of target $t_*$ given input $x_*$ and parameters; after eq. (8), paper explicitly says $x$'s are omitted from the givens of target likelihood $p(t|.)$, but I think they should have been kept there to avoid confusion. It might also be helpful to remind that $(\mathbf{x}, \mathbf{t}) = \{(x_1, t_1),...,(x_n, t_n)\}$ are the observed (input, target) pairs.
Paper says when $\mathbf{w}_{MAP}$ is calculated, which would be the same as $\mathbf{w}_{PLS}$, mean of target distribution will be the same as the deterministic target $t_* = y(x_*; \mathbf{w}_{PLS})$ from Penalized Least Squares. That is,
$\mathbb{E}[t|x_*,\mathbf{w}_{MAP}, \sigma^2] = \int tp(t|x_*,\mathbf{w}_{MAP}, \sigma^2)dt = y(x_*; \mathbf{w}_{PLS})$
In summary, in (Penalized) Least Squares, both target and parameters are deterministic. In non-Bayesian and Bayesian MAP, parameter is deterministic but target is probabilistic. In (true) Bayesian, both target and parameters are probabilistic. In full Bayesian, hyper-parameters (parameters of distribution of parameters, e.g. $\alpha$) are also probabilistic.