Hacking through the jungle with machetes, they lead us to a camp abandoned just two weeks earlier by the Lord's Resistance Army (LRA), the rebel group commanded by Kony, who claims to be fighting to install a government in Uganda based on the Biblical 10 Commandments.
A rebel defector led the Ugandan army to the camp, which would have housed between 50-60 fighters, including women and children.
Among the children was a five-year-old boy, said to be Kony's son, and his mother Doreen Abango. She was abducted by the LRA aged 13, and had managed to escape only 10 days earlier.
"I started walking at night with my child… I did not leave any tracks that could be used to follow me, because I knew if I was found I would be killed," she said.
'Strange child'
Few in northern Uganda escaped unscathed from Kony's war, including his own family.
His older sister Gabriela Lakot still lives in Odek, northern Uganda, where Kony was born.
"As soon as Kony was born he stood on his own two feet," she says. "Everyone was astonished and wondrous saying: 'This child is strange.'
"God has brought a curse on this family - that was my mother's lament. Kony brought so much trouble on us up to now. Everyone hates us."
Gabriela also blames her brother for the death of her son.
"My first born was shot by the Ugandan army when he was with Kony. All that is his fault," she said.
Kony's rebels have terrorised swathes of South Sudan, Democratic Republic of Congo and the Central African Republic over the years.
It is estimated by the Ugandan army that there are only 450 members of the LRA left.
Caesar Achellam, a senior LRA commander, was captured in May this year. He says Kony will never surrender because of indictments in the International Criminal Court (ICC).
"Kony is afraid, he is really afraid. He is afraid of the accusations against him in The Hague," he said.
Kony is wanted by the ICC for war crimes including rape, the murder of civilians and forcibly recruiting children to serve in his Lord's Resistance Army.
'Die like Hitler'
Kony's global notoriety increased earlier this year because of the internet video Kony 2012, which has been watched tens of millions of times since it was posted online by the US advocacy group Invisible Children.
Then available in the UK on the BBC iPlayer
The Ugandan President Yoweri Museveni is clear that more foreign support is needed to bring Kony to justice.
He said: "We are now an international mission. It is not a domestic mission and we cannot be an international mission on domestic resources. It is up to the international community to help."
An African Union force of 5,000 troops from the four countries most affected by the LRA was due to begin operations in March, but it has not happened yet.
Ugandan minister Betty Bigombe is one of many African leaders who have tried and failed to talk Kony into making peace.
According to Ms Bigombe, Kony himself predicted how the war would end: "He would tell me, I know exactly how I am going to die. I am going to die like Hitler.
"One day people will wake up and find that I have been dead for some time."
Panorama, Kony: Hunt For The World's Most Wanted BBC One, Monday, 20 August at 20:30 BST then available in the UK on the BBC iPlayer.
Deep in the jungle of the Central African Republic, about 2,000 Ugandan troops are spearheading the campaign to catch Joseph Kony.