Salva Kiir issued a decree dismissing all ministers and deputy ministers, as well as Vice-President Riek Machar.
No clear reason was given for the sackings, but analysts say Mr Kiir and Mr Machar have been embroiled in a power struggle for months.
South Sudan has suffered from chronic economic problems since it split from Sudan in 2011.
There was tight security outside ministry buildings on Wednesday, while radio broadcasts urged people to remain calm, the AFP news agency reports.
Its stability has been hampered by lingering rows over territory and oil with its northern neighbour.
Recently some of Mr Kiir's colleagues have hinted at discontent with his leadership.
Local reports say Mr Kiir is battling to maintain control of the Sudan People's Liberation Movement (SPLM), the former rebel group that now runs the country.
Mr Machar, who had been stripped of some of his powers in April, had hinted that he may stand against Mr Kiir for leadership of the SPLM before the next presidential election in 2015.
By Nyambura WambuguBBC News, Juba
Dissolving the cabinet in South Sudan hints at a wider attempt by the president to restructure not only the government but power and access to power in the country.
There have been reports in recent weeks of strained relations within the governing SPLM and the relationship between the president and his deputy are said to be at an all-time low.
Although the cabinet restructuring has been a long time coming, its timing is curious and raises suspicions that the president could be using his executive powers to stamp out dissent in the party and disagreement within his government.
Whether the sacking of the entire cabinet will succeed in calming the disquiet within South Sudan's political circles remains to be seen -there is also a real fear that this might further divide the country along ethnic lines.
The BBC's Nyambura Wambugu in the capital, Juba, says there is a real fear that the sacking could further divide the country along ethnic lines.
Mr Kiir is from the Dinka community, which is the largest in South Sudan, while Mr Machar is from the second-largest group, the Nuer, some of whom have complained about Dinka domination.
Under-secretaries have been put in charge of the departments and the government insists it can function smoothly until new ministers are appointed.
Barnaba Marial Benjamin, who had been information minister until Tuesday, characterised the sackings as a "major reshuffle" and said it was part of Mr Kiir's constitutional mandate.
"Some of these people will come back and some will not," Mr Benjamin AFP.
Among the other leaders to be dismissed was Pagan Amum, the top negotiator in peace talks with Sudan.
He was removed from his post as SPLM secretary general and the decree said he would be investigated for mismanaging the party.
Last month Mr Kiir sacked two other senior ministers embroiled in a multi-million dollar financial scandal, a decision reportedly criticised by Mr Amum.
It was unclear whether Tuesday's decree had any link to the scandal.
South Sudanese journalist Nhial Bol said Mr Kiir had probably acted to end the paralysis in his government.
"Things have not been moving in the government because of this internal fighting over who is going to control the SPLM," Mr Bol told Reuters news agency.
South Sudan is rich in oil, but is one of Africa's least developed countries, with few paved roads and poor health and educational facilities.
It gained independence from Sudan in July 2011 after a decades-long civil war.
However, the oil refineries are on Sudanese territory, so the South relies on its northern neighbour to get its product on to the market.
South Sudan's president has sacked his entire cabinet, in an apparent power struggle with other senior leaders.