Stabbing victim John Culshaw's remains were retained by Greater Manchester Police without his family's knowledge.
Dozens of other families have also been told their loved ones' body parts were kept by the force.
GMP said whether or not to contact the affected families was a decision it "agonised over for a number of months".
Mr Culshaw, 26, was stabbed to death in Wigan in 1993.
His family buried him believing his body was intact, and were unaware his stomach, liver and other tissues had been retained after two post-mortem examinations until police officers visited them in September last year.
His mother, Jennifer Shaw, said she wanted to know why it had taken so long to come to light.
"He's my son. And he's been left - half of him," she said.
"If he'd have died and they'd asked me if they could use the parts to help somebody, then yes. But just to be sat in a lab for 23 years doing nothing, that's just horrendous."
The body parts were discovered following an audit of the former Forensic Science Service in 2014, the Manchester Evening News revealed.
It found more than 180 samples - including major organs - from victims of crime in Greater Manchester were being held for the police force, which has admitted there could be more body parts unaccounted for.
It became illegal in 2006 to remove or store human tissue without families' consent following an investigation at Liverpool's Alder Hey hospital in 1999.
Organs from hundreds of babies who died at the hospital were secretly taken.
The body parts kept for Greater Manchester Police were removed before that date.
Assistant Chief Constable Debbie Ford said: "Once we knew about the samples we felt that such a sensitive issue was never going to be solely about following the regulations or the letter of the law.
"For this reason we consulted with a number of relevant people, including community and faith groups, as well as our own Ethics Committee. Following this extensive consultation it was agreed we had an ethical and moral duty to inform the next-of-kin in the vast majority of cases."
She said officers had offered a range of options for "sensitive disposal" of the samples, which the force would pay for.
However, families have been told some samples could remain unaccounted for "for various reasons", she said.
John Culshaw's family will bury his organs in a private second ceremony on Thursday.
"Somebody has made a big mistake. Not just me but a lot of other families are suffering as well," Ms Shaw said.
"I don't want anyone else to go through this. It's devastating."
A mother is preparing to bury her son for a second time after being told his organs were removed and stored in secret for more than 20 years.