Why do we produce so many sperm?
The term you should look up is sperm competition, there is an introductory wikipedia article about it here, where you can get the basic picture and get some references to explore.

Simply said, males often compete with other males in the reproductive tract of the female, thus a higher number of sperm will lead to a higher chance of fertilizing the female in competition with other males - leading to an arms-race of the number of sperms a male eject into the female.

There is a lot more to it, of course, but I believe this well explains the most fundamental driver of why males generally produce and eject so many sperms.