Can devices be added in parallel indefinitely?
In an ideal circuit, with an ideal voltage source, and ideal wires, then sure. You could add as many devices in parallel as you want.

Batteries are not ideal voltage sources. As the current through a battery increases, the voltage across it will decrease, and the power it converts to heat will increase. At some point either the voltage will be too low for your loads, or the battery will overheat and catch fire. (Some types of batteries will merely get warm or hot, and not catch fire, even when shorted)

Additionally, a battery will only hold a certain amount of energy. If you make it supply more power (more energy per unit of time) then it won't last as long.

Wires also have some resistance. As the current through a wire increases, the voltage across it will increase (which means less voltage across the load), and the power it converts to heat will increase. The same dangers apply - it's possible for the voltage drop to cause the voltage across the load to be too low, and it's possible for the wire to get so hot that it burns the insulation or even melts.

Note that all of these are related to the total amount of current supplied, not the number of devices in parallel, except to the extent that connecting things in parallel increases the current. Connect a million 100-megaohm resistors in parallel across a battery and not much will happen. Connect a single 1-ohm resistor and you might have problems.