The Cave that the garden surrounds was first discovered during the 1845-46 survey of the area, with the fresh water supply in the cave being an attraction that initiated the first settlement of the area.
In 1922 a captured German gun from WWI was added to the garden, however this was later removed.