Jamal Kiyemba, a Ugandan, was arrested with the the help of US officials in the capital on Tuesday, police added.
Prosecutor Joan Kagezi was shot dead on 30 March, a day before the trial of men accused of involvement in bombings.
Mr Kiyemba, a former UK resident, was freed from Guantanamo Bay in 2006 without being charged with any offence.
For the latest news, views and analysis see the BBC Africa Live page.
He was arrested in Pakistan in 2002 for alleged links with al-Qaeda, and later transferred to the controversial US-run detention centre in Cuba.
"There was an operation which we carried out with our counter-terrorism team because we suspect that Jamal Kiyemba and his colleagues have been involved in some form of criminality," Ugandan police spokesman Fred Enanga told Associated Press news agency.
There was no conclusive evidence to link him to Ms Kagazi's killing, but he was being questioned about his possible role in her murder and in other crimes, he added.
Gunmen on a motorbike shot dead Ms Kagezi in a suburb of Kampala.
She was prosecuting 13 men accused of being linked to suicide bombings which killed 76 people in Kampala in 2010.
The US deported Mr Kiyemba to Uganda after the UK refused to take him back.
He had lived in the UK for eight years before travelling to Pakistan.
US President Barack Obama has promised to shut the Guatanamo Bay detention centre because of concerns of widespread human rights abuses.
An ex-Guantanamo Bay detainee has been arrested in Uganda for questioning over his possible role in the killing of a top prosecutor, police say.