ESP8266 loses wifi connection on switching a load
I solved the problem by rewiring. Everything is still wired according to the diagram above but with the following changes:


Moved the step down regulator further away from the motor. (It is now below the esp due to a lack of space
Rotated the esp and moved it into the edge as far as possible. This way it is further away from the motor and the wires to the opto-isolator as well as the wire to the light sensor is not as close to the antenna as before.
Reduced the amount of wires by connecting everything (opto-isolators and step down converter) to the nearest ground point of the rollotron board instead of running all wires to the bottom and connect them there


What i don't know is, which measure was the important one.

Tip: Don't waste your time with grounded shielding. It is not worth it.
I stumbled across another topic: If you create a DIY project with a motor ensure that you have a capacitor across the terminals of the motor. You can also add capacitors from both terminals to the case to reduce noise. (The rollotron uses such a technique already)

Here is a picture after rewiring: