Geldner was born in Saalfeld, Saxe-Meiningen, where his father was a Protestant clergyman.
Geldner studied Sanskrit and Avestan at the University of Leipzig in 1871 before moving to the University of Tübingen in 1872.