Some of the rock in the ground is made of limestone. Over long periods of time rain can hit the limestone. And cause a acidic reaction. The limestone will slowly disintegrate. A large gap with be where the limestone was. Result in the cave. 
How do caverns form?