LEDs dim as more turn on
In my confusion as to why the highly complex PCB assembly wasn't working, I almost completely forgot about the little breadboard in the corner hooking all the power lines together. I mean, it's only connecting a few wires together, right? There's not even any components on it, it's just tying some wires together.

On closer inspection, the part having the problem was connected to the furthest point on the breadboard away from the power supply. So I moved the wires to be right next to the incomming power, and the problem seems to have disappeared. Still not sure why exactly, but it seems to fix my issue.