Light changes speed as it moves from one medium to another (for example, from air into the glass of the prism).
Prisms are sometimes used for the internal reflection at the surfaces rather than for dispersion.