Why is the radial distribution at the nucleus 0 but not the radial wave function?
You state 'but we already know from the radial wave function that there is a non-zero probability of finding an electron in an infinitesimal volume at the nucleus.'
To add to Poutnik's answer, I think that this sentence states the misunderstanding. The radial part of the wavefunction is $R_{n\ell}(r)$ and when normalised
$$\int_0^\infty R_{n\ell}^*(r)R_{n\ell}(R)r^2 dr=\frac{4}{a_0^3}\int_0^\infty e^{-2r/a_0}r^2 dr=1$$
where the $r^2$ comes from the volume element needed to integrate over all angles which is $r^2\sin(\theta)drd\theta d\phi$. The probability from $r\to r+dr$ is then for the 1s orbital $\displaystyle p=\frac{4}{a_0^3}r^2e^{-2r/a_0}dr$ whereas the 1s wavefunction itself has the form $\sim e^{-r/a_0}$. For other orbitals, e.g $2p_{\pm 1}$ the wavefunction is a function with complex numbers so cannot represent anything physical.