The plan for the building was put forward to the city council on 28 January 1773 by the governor Felipe de Fondesviela y Ondeano, marqués de la Torre.
The last of the Colonial governors vacated the palace in 1898 when Cuba gained independence under the terms of the Treaty of Paris, and it was used by the US military governors from 1899 until 1902.