 The castle sits on Collegian Hill , the former site of Canton Iron Works industrialist , Horace Abbott with its Victorian @-@ styled twin mansions known as " <unk> " and " Woodlands " built in the 1870s and later known as the " Gilman @-@ Cate Estate " in the early 20th Century when it passed to his children . Nearby also was the historic " Montebello " estate of the 1790s of General Samuel Smith , ( 1752 @-@ 1839 ) , a noted American Revolutionary War hero and officer in the War of 1812 in the Battle of Baltimore , commanding the Maryland state militia forces against the British Army and Royal Navy attack . Smith also served as a local merchant , elected U.S. Representative , and Senator , and near the end of his remarkable eight @-@ decades life , served as the Mayor of Baltimore . 
