What is the technical difference between a diffuse shader and a rough glossy shader?
A glossy shader at "1" roughness is still far more "smooth" at the microscopic level than a true diffuse shader. This intrinsic microscopic hardness or smoothness stops it from having much, if any diffusion reflectance, and it has to rely on it's specular reflectance, which transmits reflections from nearby direct and indirect light sources.

Diffuse materials are usual highly porous or permeable at the microscopic scale, so they trap and scatter any light that hits them at unpredictable angles, and scatter it almost equally in all directions. Likewise, a lot of their "albedo color" comes from underneath the shallow surface. Metal, some liquids and some amorphous solids have no "diffusion" ability due to the hardness and solid nature of their microscopic surfaces, so all the light that hits them immediately bounces off in a predictable, angualar direction.

The technical explanation is that the glossy shader is still reflecting light at (mostly) coherent and predictable angles in comparison to the diffuse, which is spraying light every which way from sunday.

As a rule of thumb, you use "pure" glossy shaders for metals, some liquids and some amorphous solids (like varnish)

Diffuse is for really rough and microscopically porous things like plaster, rock, concrete, etc