Does increasing mAh reduce voltage drop?
I have some WS2812B LED strips and instead of hooking them up in parallel, I did them in series. 


What you mean is that, while the LEDs are in parallel within the strips, you have series connected or 'daisy-chained' the strips connecting the + and - out of one to the + and - input of the next. (Data line has also been connected through.)


  The lights just get dimmer and dimmer as they go on. This was my mistake.


Yes. The resistance of the strips will cause a voltage drop along it. The longer the circuit the more voltage drop.


  I have a 5v 20,000mAh battery. If I were to change the battery, say to 5v and 30,000mAh or higher, would this help with the voltage drop? 


No. If you measure the voltage at both ends of the strips you will probably find that the input voltage is close to 5 V but that the voltage at the far end is significantly less. The mAh rating tells you how long the battery will last.


  Would more current mean more LEDs would become brighter?


No. You want a fairly constant voltage across all the strips.

You need to run power wires out to each strip or pair of strips. You should probably daisy-chain two or at most three strips in any section. You can feed power in both ends to even out the brightness. Observe correct polarity.

I have written a little more on the topic in Flexible LED strip light.