That specific hill (mount) is one of the many squat hills that form Jerusalem, which also includes Mount Moriah (the Temple Mount), the Mount of Olives, etc. Over many centuries, until as recently as the Ottoman era, the city walls of Jerusalem were rebuilt many times in new locations, so that the particular hill known as Mount Zion is no longer inside the city wall, but its location is now just outside the portion of the Old City wall forming the southern boundary of the Jewish Quarter of the current Old City. Most of the original City of David itself is thus also outside the current city wall.