Mr Ordonez said Gen Palomino would be investigated over the alleged creation of a male prostitution ring.
The general is also under investigation for alleged illicit enrichment and conducting illegal wiretaps.
Gen Palomino welcomed the probe, which he said he knew would clear his name.
"I hope that the justice system can restore the dignity which has been taken from me through lies and infamy," the general said in a statement.
The investigation was triggered by allegations by a police captain who said he had been abused by higher-ranking officers during his time as a cadet.
Mr Ordonez said that allegation had been backed up by a complaint by a now retired police captain.
According to the complaint, young male police cadets were cajoled and threatened into having sex with higher-ranking officers.
Mr Ordonez said that according to the testimony gathered by his office, a senator had also allegedly been implicated in the prostitution ring, called "Community of the Ring".
He also said that the alleged incidents had taken place "with the help and complicity of police officials, including the director general [Rodolfo Palomino]".
The wiretapping accusations also relate to the case. Allegedly police officers tapped the phones of journalists investigating the scandal.
Gen Palomino has strongly denied any wrongdoing.
He has also denied allegations of illicit enrichment and has provided prosecutors with documents which he says show where his money and properties come from.
He says the allegations are a political vendetta designed to drive him out of office.
Colombia's prosecutor general, Alejandro Ordonez, says he will open a "disciplinary investigation" against the country's chief of police, Gen Rodolfo Palomino.