The north side of Albany Hill was long used by Native Americans, as evidenced by large mortar stones and remnants of a shell midden.
Beginning in the late 19th century, Judson Powder Works manufactured dynamite at the northwest foot of the hill.