Using 5 12V DC motors
Q1: Can I connect them directly to the power supply and activate them using a bistable switch (while using flyback diodes)?

Yes. You can switch them with one big switch or five smaller switches (mechanical or transistor). Effectively you are connecting them all to the supply.

Q2: Or should I use an arduino, 5 MOSFETS, flyback diodes (I have a lot of 14N007) to slowly (with PWM) start each one?

You are unlikely to be starting them on load but your test measurements indicate that you may still reach supply capacity when all are started simultaneously. This may or may not be a problem. I wouldn't recommend the complexity of PWM control unless you think you might need speed control in the future. A simpler alternative would be to sequence the motor starts 1 - 2 - 3 - 4 - 5 with, say, 0.25s between them.