5V Power Supply oscillates once another consumer is turned on
I initially wanted this as a comment but it seems I don't have enough reputation for this.

A few questions that might help cornering the issue : 


How much power does your fan require? Have you tried measuring the current it draws when powered on? Is it significant compared to the max Amperage you can have on your 230V? (I shouldn't think so but maybe your fan is some kind of huge thing).
Have you tried replacing the fan with something of similar power needs (or higher) to see if the effect is the same? Things like a boiler, a toaster or a vacuum cleaner could be good tests since they can draw quite a lot of current.
To see if the problem is your 5V power converter, when the fan is not running, is the output really a "smooth" 5V? Could you measure the AC input of the 5V converter when the fan is on? Just to see if the peak-to-peak value of your 230V AC decreases when the fan is on.
You say that when your turn on the fan, the Ambilight flickers for a moment. Does it mean that after some time, even if the fan is still on, the 5V goes back to a normal behaviour?