What is the physical-based/theoretical reason why supersonic area ruling works?
That's because Bernoulli's principle breaks down at and above supersonic velocities.

In the subsonic realm, when an atom is sped up, it doesn't have much time to spend pushing around a fixed place (it passes by that place quickly). In doing so, the static pressure drops as the speed increases. And in doing so, the neighboring atoms, who are also not doing as much repelling, can group together.

In Busemann's and Whitcomb's metaphor, the tube of air gets thinner.

In supersonic flow, much like the reason behind shockwaves, the communication between atoms breaks down. They are moving too fast for the air to carry out the effect of moving fast (pressure disturbances cannot propagate), and thus, the tube of air doesn't get smaller. This results in an increase in the drag versus the calculated one; instead of the tubes getting smaller, they remain both big and fast.

Drag increase is highest when there is a sudden cross sectional change. Consider the face of a box compared to a cone pointy-end first.

Applying that, the supersonic area rule aims at keeping the cross sectional size of the whole vehicle constant without abrupt changes. If thinning the fuselage where the wing meets it is not possible, then gradually thickening it before the wing starts is also an option.

The tubes not getting smaller, and considering the cross section of the vehicle as a whole, were the reasoning behind the area rule.



Reference and further reading: Chapter 5 - The Whitcomb Area Rule - From Engineering Science to Big Science, via NASA.