George Boldt, general manager of the Waldorf-Astoria Hotel in New York City and manager of the Bellevue-Stratford Hotel in Philadelphia, and his family enjoyed an earlier frame cottage on Hart Island (the island's original name) for several summers, which they greatly expanded.
The construction of Boldt Castle ceased abruptly in early 1904 after the death of Boldt's wife, Louise Kehrer Boldt.