Why does an element's chemical properties rely only on its valence electrons and not on anything else?
Technically, other electrons could play a part in determining chemistry. However, there is a significant energy barrier between the p-orbitals of one period and the s-orbital of the following period.

Consider copper. The characteristic deep blue colour of $\ce{[Cu(NH3)4(H2O)2]^2+}$ derives from the absorption of photons whose energy corresponds to the difference of the $\mathrm{3d}_{z^2}$ and $\mathrm{3d}_{x^2 - y^2}$ orbitals. This is spin-allowed because copper(II) is a d⁹ system. The wavelength corresponds to approximately $610\:\mathrm{nm}$.

On the other hand, the energy between a $\ce{2p}$- and a $\ce{3s}$-orbital can be measured using x-ray transition — it corresponds to a photon of a wavelength of approximately $1\:\mathrm{nm}$. This is about six-hundred times the energy difference compared to inside the d-orbitals.

Of course, this is a slightly unfair comparison. The energies between $\ce{3d}$ and $\ce{4s}$ are noticeably larger than inside of the $\ce{3d}$ orbitals, and the energy difference between $\ce{4s}$ and $\ce{4p}$ should again be well larger — but they will remain significantly smaller than energy differences across shells. The former are typically visible or UV, the latter are typically x-ray.