The Governor 's office is now in the historic Chief Secretaryâ s Building at 121 Macquarie Street . The Governor 's present day office is the historic Chief Secretaryâ s Building nearby , at 121 Macquarie Street .