 Hunt remained at Netley until 1857, by when the castle had an oriel window overlooking the sea, and the battlements had been blocked in to allow the construction of a second storey to the building.George Sherriff leased the castle between 1868 and 1873, building a stone wall around the front of property.Sir Henry Crichton bought the castle in 1881, along with the surrounding gardens, orchards, pond and a boat house.