Understanding offline switcher circuit
Yes, you will get some noise from the AC on the DC output leads. But the high frequency AC voltage between the AC and DC leads is significantly reduced due to the shunting effect of the cap. This can and does reduce radiated EMI. This cap is often used and is accepted despite its drawbacks.

Flyback converters like this are simple and cheap and therefore popular. They require tight coupling between the transformer primary and secondary. Tight coupling means low leakage inductance which is often achieved by sectionalised windings. Interwinding capacitance gets worse when you section the windings more. It's possible to use shields to eliminate the need for the cap but this increases cost and size of the transformer.