Is it realistic to have a natural oxygen source in a cave?
According to this page, the CO2 buildup in the cave might also be a problem. 

But, if there was a river that ran on the surface, then oxygen and nitrogen would dissolve into the water. The stream could then pass into the cave and release some of the dissolved oxygen into the oxygen deficient environment in the cave. It would also remove the CO2. Of course, if the surface of the planet is frozen, then the river would have to avoid freezing somehow. But perhaps there is a hot spring on the surface. 

My intuition is that the rate of oxygen dissolving out of the water would be very slow and it could be increased by increasing the surface area of the water. In the human lungs, there are many, many tiny branches (Alveoli) that allow the lungs to have a huge surface area so that the slow diffusion of oxygen from the air into our blood can go fast enough for us to survive. Perhaps the cave-people can accomplish something similar if they let their oxygen-rich stream pass over a tall, broad waterfall that would cause the river to break up into tiny droplets, thereby increasing the surface area. This has the added benefit of looking super cool in case your story gets made into a movie :)

Other options: 
1) they could generate electricity and use that to make hydrogen and oxygen from water. 
2) http://www.mpg.de/621120/pressRelease201003241?filter_order=L