Atmega32u4 USB 2.0 Routing Check
According to this article avoid routing the differential USB signal over the BGA pins


Do not place probe or test points on any high-speed differential
signal.
Do not route high-speed traces under or near crystals, oscillators,
clock signal generators, switching power regulators, mounting holes,
magnetic devices, or ICs that use or duplicate clock signals.
After BGA breakout, keep high-speed differential signals clear of the
SoC because high current transients produced during internal state
transitions can be difficult to filter out.
When possible, route high-speed differential pair signals on the top
or bottom layer of the PCB with an adjacent GND layer. TI does not
recommend stripline routing of the high-speed differential signals.
Ensure that high-speed differential signals are routed ≥ 90 mils from
the edge of the reference plane.
Ensure that high-speed differential signals are routed at least 1.5 W
(calculated trace-width × 1.5) away from voids in the reference
plane. This rule does not apply where SMD pads on high-speed
differential signals are voided.
Maintain constant trace width after the SoC BGA escape to avoid
impedance mismatches in the transmission lines.
Maximize differential pair-to-pair spacing when possible.