Does exponentiation reduce to multiplication or the other way around?
Both statements are accurate, assuming $\le_p$ refers to polynomial-time Turing reductions (i.e., Cook reductions), and assuming that in the exponentiation problem, the exponent is specified in unary (otherwise you can't exponentiate in polynomial time).  Given an algorithm to multiply in polynomial time, you can use it to exponentiate in polynomial-time via a Turing reduction, since $a^k = a \times a \times \cdots \times a$.  Also, given an algorithm to exponentiate in polynomial time, you can use it to multiply in polynomial time via a Turing reduction, since $ab = [(a+b)^2 - a^2 - b^2]/2$.  That said, since both multiplication and exponentiation can be computed in polynomial time directly, there's really no need for such a fancy reduction: it's immediate from the definitions that any problem in P reduces to any other problem in P.

You might want to refresh your understanding of the definition of the reductions.

Note that we usually apply reductions to languages, i.e., to decision problems.  So, to really understand how to apply the formal theory to your question, you'd first need to re-cast it as a question about a formal language or about a decision problem.  "Multiplication" is not a decision problem.  Re-casting as a decision problem will complicate things a little bit, and change the nature of the reductions (for instance, the reductions I hinted at above might no longer work), but the bottom line will still remain true: since you can reduce any problem in P to any other problem in P, there will be a reduction in both directions.

See also What is the definition of P, NP, NP-complete and NP-hard?, What are common techniques for reducing problems to each other?, What is the difference between an algorithm, a language and a problem?, and NP-completeness: Reduce to or reduce from?