Why is an atom with excess neutrons unstable?
Pulling out Theoretical Nuclear Physics by Blatt and Weisskopf (published in 1952, so a bit dated in areas but still a good introduction), one finds (in Chapter VI.1 The Systematics of Stable Nuclei) that the stability of isobaric nuclei (all the same $A$) is determined by:

(1) The symmetry effect: the charge independence and the exchange character of nuclear forces, together with the Pauli exclusion principle, strongly depresses the energy of nuclei with equal or nearly equal numbers of protons and neutrons.


(2) The charge effect: the effect of the Coulomb repulsion of the protons favors nuclei with fewer protons than neutrons. The neutron-proton mass difference also enters the charge effect, but only as a small correction. The charge effect increases in importance with increasing nuclear charge.


(3) In a few special cases a third effect is important: the spin dependence of nuclear forces favors parallel spin over anti-parallel sin of a pair of (extra) nucleons.

So, yes, you picked up on the Coulomb bit (number 2 above), but it is the symmetry effects of the strong force that are more important, particularly for light nuclei. Until one gets to some nuclear physics, the characteristics of the weak and strong forces just aren't familiar to us humans.