Solid State Relay "fading" instead of dropping secondary voltage
Your SSR includes a snubber on the AC side that can pass up to 3.0 mA of leakage current in the "off" state. Your USB power adapter only draws something like 20-25 mA even at full load. (I'm assuming it's a fairly efficient switching converter.)

When you switch off the drive to the SSR, the leakage current allows the AC voltmeter to read essentially full voltage, clipped only by the remaining voltage on the USB adapter's input capacitor. As this capacitor discharges into the load, the waveform gets clipped at lower and lower values, until it reaches an equilibrium where the 3 mA matches the residual current draw of the adapter.

Looking at the AC waveform with an oscilloscope would make this process very obvious &mdash; but be very careful about how you hook it up. Do NOT connect the scope's ground lead drectly to either side of the power line; instead, use an isolation transformer or use a 2-channel scope in differential mode.

Connecting the lamp as a parallel load "soaks up" the leakage current, preventing it from driving the voltage up to the clipping level imposed by the USB adapter. The primary of a small power transformer (leave the secondary unconnected, the magnetizing current alone will be enough) would work just as well, without the excessive dissipation when "on".