Why is Price Vector Orthogonal to Vector connecting two bundles on Budget Hyperplane
Note that $p$ is not orthogonal to consumption vectors on the budget line, but it is orthogonal to a any vector $v$ that satisfies $x+v=x'$ with $x,x'$ in the budget line. MWG are drawing vectors starting from some $x$.

About the slope: In the budget line you can see that the slope for any $x$ is $D_x (p.x)=p$ (that is the line that is drawn from $x$). That line or vector can be represented with the function $x_2=(p_2/p_1)*x_1$ (note that if $x_1=p_1$, $x_2=p_2$). The slope is $p_2/p_1$.

On the other hand, all vectors on the budget line satisfy $x_2=(-p_1/p_2)*x_1+w/p_2$. The slope here is $(-p_1/p_2)$. This slope is the rate of exchange. Both slopes imply that the two functions (the vector $p$ and the budget line) are orthogonal. Or maybe it's better to say that the orthogonality between the budget line and the vector $p$ imply that $dx_2/dx_1=-p_1/p_2$, that is, the last term captures the rate of exchange.