MCP1755S regulator and output voltage
This family of LDOs is highly sensitive to output and input capacitance. Low ESR is mandatory.

Changing for ceramic capacitors makes it work a lot better (my ATmega initializes and accepts code upload). There remains many ways to improve stability, according to the document AN765 - Using Microchip's Micropower LDOs:


  However, the ESR of ceramic capacitors sometimes can be too low, requiring a 1&nbsp;Ω series resistor to ensure stability.


I'll order a resistor.