At higher altitude, the air pressure is lower and therefore the pressure inside the balloon is lower.
A different approach for high altitude ballooning, especially used for long duration flights is the superpressure balloon.