UN chief Ban Ki-moon told reporters that he had requested his resignation.
It comes a day after Amnesty International alleged that a 12-year-old girl was raped by a UN peacekeeper.
The 10,000-strong UN force, deployed last year to help restore order in CAR, has also faced allegations of sexually abusing street children.
In June, Mr Ban set up an independent review panel to examine the UN's handling of allegations of sexual exploitation.
Earlier, the UN had denied allegations it covered up child abuse by French troops also serving in CAR.
BBC Africa Live: News updates
"I cannot put into words how anguished and angered and ashamed I am by recurrent reports over the years of reports of sex abuse and exploitation by UN forces," Mr Ban said when he announced the resignation of the Senegalese army general.
"I will not tolerate any action that causes people to replace trust with fear."
The BBC's UN correspondent Nick Bryant says the UN chief also noted that the problem went beyond one conflict and one person.
He is convening a special session of the UN Security Council on Thursday to discuss what he called "the scourge of sexual exploitation".
Mr Ban will also hold a video-conference call bringing together the force commanders of all the peacekeeping operations around the world to underscore their responsibility to uphold the values of the UN, our correspondent says.
The UN mission in CAR is known as Minusca, from its French acronym, and has been headed by Gen Gaye since it took over from a smaller African Union force in September 2014.
Unprecedented violence escalated in March 2013 when mainly Muslim rebels seized power and the country descended into ethnic and religious violence, with tens of thousands of people fleeing their homes.
This week, Amnesty International also accused UN soldiers of killing a 16-year-old boy and his father during an operation in the capital, Bangui.
The UN envoy to Central African Republic (CAR), Babacar Gaye, has been sacked amid multiple allegations of sexual abuse by peacekeepers.