Noise on the whole board after USB connection
Maybe at Your test suite You can't observe any bad behavior after connecting the USB cable, but some day someone will connect Your device to her/his PC with very bad and ugly power supply... In EMC high immunity is as important as low emission.

Good practice in designing USB and similar serial interfaces is to use common mode chokes specially designed for fast data lines. Below You can see such common mode choke together with ESD supression and single choke on supply line.



Example common mode chokes for USB are WE-CNSW series from Wurth or DLM series from Murata.