This was a key rebel demand during drawn-out talks which led to a peace deal in August. All foreign forces were meant to have left by 10 October.
Meanwhile, a rebel leader has told the BBC the deal is unlikely to work and said he may take up arms again.
Tens of thousands of people have died since the civil war began in 2013.
Uganda's defence chief Gen Katumba Wamala told the press that all soldiers should be out by 1 November.
Minister of State for Foreign Affairs Henry Okello Oryem added in a statement that Uganda's intervention "helped to stop what was likely to be the worst genocide in the region".
Uganda has not revealed how many troops it has in South Sudan, where they are supporting the government.
But they are stationed in three bases: The international airport in Juba; in Nesitu just south of the capital and at their biggest base in the town of Bor, which is hosting some 3,500 soldiers.
The BBC's Catherine Byaruhanga in Uganda says that a neutral regional force is meant to step in once Uganda withdraws but it is not clear when that will happen.
Senior rebel leader Gen Johnson Oloni said he might resume fighting because of President Salva Kiir's recent decree creating 18 new states.
He called the plan a "naked power-grab", the BBC's Tim Franks reports from South Sudan.
He also said the government was undermining the deal with attacks on civilians.
The rebel leader said land appropriations had pushed his ethnic group to use force in the past and they could do so again.
Gen Johnson is head of the Shilluk militia, which has been accused by the UN children's agency Unicef of abducting hundreds of boys and forcing them to become child soldiers.
The US had proposed that he should be subject to a travel ban and assets freeze, along with an army chief, but this was blocked by Russia and Angola at the UN Security Council.
South Sudan's elusive peace:
Five obstacles to peace in South Sudan
Men of dishonour
South Sudan gained independence from neighbouring Sudan four years ago making it the world's youngest country.
Two years later, in December 2013, a conflict erupted after President Salva Kiir accused Riek Machar, his sacked deputy, of plotting a coup.
Mr Machar denied the allegation, but then formed a rebel army.
Much of the fighting has been carried out along ethnic lines, between Mr Machar's Nuer group and Mr Kiir's Dinka, the two dominant ethnic groups in South Sudan.
Ugandan troops will start leaving South Sudan by the end of this week, according to the head of the Ugandan force in the country.