In 1839 it was chosen as the site of prison by the Governor of the colony of New South Wales , Sir George Gipps . In 1839 it was chosen as the site of a new penal establishment by the Governor of the colony of New South Wales , Sir George Gipps .