Should not specular reflection on a rough surface be similar to diffuse reflection? Different BRDFs are usually used to compute diffuse and specular reflection.
Some of the most often used include for example the Lambert BRDF for diffuse reflection and the Cook-Torrance BRDF for specular reflection.
Cook-Torrance BRDFs are parametrized by the surface roughness: the rougher the surface, the "blurrier" the reflection.  

From a conceptual point of view, should we consider that for an extremely rough surface, the specular reflection must look like a diffuse reflection? Can there be a continuity between both components of the reflection?
No, because the underlying physics is not the same, nor the lobe shape - not to speak of their parameters such as color and Fresnel term.

Specular is really true surface interaction with the interface material/air, so it has Fresnel modulation and the internal medium has no influence on colors. But the surface condition strongly influence the reflectance, of course. Think of it as the reflections on ocean surface.

Diffuse is due to the subsurface scattering, light entering the medium, and thus gains the color characteristic and loose directionality. Think of it as the green color of the water turbidity. If depth of penetration is less than pixel size CG people don't call it "subsurface" but the physics is the same. 

Of course diffuse input is what specular let pass, which is angle (and polarization) dependant. 

Beside, there exist transparent medium with internal interfaces (e.g. thin shells), so case exists were light goes inside but still acts specularely (or even wavely) and not diffusely. Oppositely, most plastics and paints/varnishs and biological medium are intrinsically transparent, but contain pigments (often based on a metal atom) that cause the opacity (diffusion or specular on these "objects in the object"). Think of them as the fishes in clear water :-) .