Downward self-reducibility of factorization
The state of the art here is: We can decide primality in polynomial time, but the fastest, general-purpose algorithm to $\underline{\rm find}$ the factors of an n-bit composite integer takes time $\approx 2^{n^{1/3}\log^{2/3}n}$.

More to your question, a primality test is the same thing as a compositeness test. Therefore, we can easily implement the 'primality decision oracle' required by any (supposed) Cook reduction from factoring to itself.

The reason, then, that factoring is commonly presented as an example of a problem that is "unlikely to be downward self-reducible" is because of the possibility that such a reduction identifies prime factors of the original integer. This can be phrased in many ways, but as Joshua points out in comments above, a good example is a reduction that takes $(n, k)$ (is there a factor of $n$ that is at most $k$?) to $(m, k')$ where $m|n$.

A reduction of this particular type would have the consequence of breaking the plethora of cryptosystems based on the hardness of factoring (without the need for a quantum computer), e.g. RSA.