How did I mess up this wiring?
I can think of a few things that would cause this.  The first is just a bad connection with either a hot or neutral - maybe a wire wasn't all the way in a wire-nut or had lots of corrosion on it. Also possible (if you trimmed any wires the second time) that a wire had a big nick in it.

The second possibility is an open neutral where one of the neutrals was not at all connected.  This can cause some circuits to have a higher voltage and others a lower voltage - hence the dim light and brighter second light.

As someone pointed out in a comment it is also possible that you wired it in series instead of parallel though I don't think this would cause a light to get brighter than normal.

Without seeing the issue occurring its really hard to tell for sure though.