How do anaerobic bacteria produce ATP for their own cellular activities?
The product is 4 ATP not 2, 2 is the net product because two are used to start the reaction.

Looking at the complete glycolysis reaction helps. 



ATP can be used to start another round of glycolysis which yields 4 ATP which can then start 2 rounds of glycolysis which yields 8 ATP, rinse, repeat. It is an exponential process, ATP can be taken from this for other processes since as long as you still have 2ATP around to can restart the exponential cascade. For example from the yielded 8 if you take half for other uses, you still get 8 ATP from the next round of glycolysis reactions.