Mr Gordhan will be replaced by Malusi Gigaba, said a statement issued late on Thursday by the preisdent's office.
Earlier this week, President Zuma recalled Mr Gordhan from planned events in the UK and US.
Sfiso Buthelezi will become Deputy Finance Minister, replacing Mcebisi Jonas.
"I have directed the new ministers and deputy ministers to work tirelessly with their colleagues to bring about radical socioeconomic transformation and to ensure that the promise of a better life for the poor and the working class becomes a reality," President Zuma's statement said.
Last October, Mr Gordhan was charged with fraud - but the charges were later dropped.
He has described the allegations as politically motivated.
Mr Gordhan has been seen as standing up to President Zuma in cabinet and has warned against corruption becoming rampant.
The South African Communist Party, an ally of the governing African National Congress, had earlier lodged a formal objection to plans to dismiss Mr Gordhan, who is widely respected internationally.
Many senior ANC figures also opposed the finance minister's removal.
South Africa's President Jacob Zuma has dismissed Finance Minister Pravin Gordhan after days of speculation that rocked the country's markets,