He was addressing thousands of people in the capital, Kigali, as Rwanda began a week of official mourning to mark the 20th anniversary of the genocide.
Many people were overcome by emotion during the ceremony, with some suffering fits.
At least 800,000 people - mostly ethnic Tutsis and moderate Hutus - died at the hands of Hutu extremists.
The killings ended ended in July 1994 when the Rwandan Patriotic Front (RPF), a Tutsi-led rebel movement, marched into Kigali and seized control of the country.
Rwanda's President Paul Kagame and Mr Ban lit a torch which will burn for 100 days - the length of time the genocide lasted.
A diplomatic row has prompted Rwanda to bar France's ambassador, Michel Flesch, from attending the event, AFP news agency reports.
By Charlotte AttwoodBBC Africa, Kigali
It began with just one, and then scores of chilling screams echoed around the stadium.
It may be 20 years down the line, but for many here, the wounds are still too fresh to bear. Paramedics went up and down the terraces, comforting and removing those who could not cope with the memories. I saw one man being carried out during the dramatisation of the recent history of Rwanda.
The play itself was a very clear depiction of the government's interpretation of what caused the tragic episode in the country's history.
Half-way through, a jeep drove in carrying the "colonialists" who marched onto the stage. They swapped their colonial straw hats for the blue helmets of the UN and then deserted the people, leaving a bloodbath in their wake. Soldiers - representing the governing RPF - troop in to rescue a devastated population.
The week of mourning began with a wreath-laying ceremony at the national genocide memorial, followed by the lighting of a flame at the Amahoro Stadium in Kigali, where UN peacekeepers protected thousands of people during the genocide.
The torch has been carried across the country for the past three months, visiting 30 districts and passing from village to village.
Thousands of people packed the stadium, having queued for hours through the fog, reports the BBC's Charlotte Attwood from the scene.
Many of them reacted with uncontrollable emotion to the stories, speeches and performances recalling the genocide, our correspondent says.
Some of them had to be led out of the stadium while others had fits, she adds.
Traditional mourning songs were broadcast over the sound systems.
There was also a dramatisation of Rwanda's recent history, which our correspondent says was a clear depiction of the government's interpretation of events.
In the play, a jeep carrying "colonialists" arrives, who swap their straw hats for UN blue helmets. They then desert the people, who are saved by the governing RPF.
UN personnel in Rwanda during the genocide showed "remarkable bravery", Mr Ban told the crowd, according to AP.
"But we could have done much more. We should have done much more," he said.
"In Rwanda, troops were withdrawn when they were most needed," he added.
"One year later in Srebrenica, areas proclaimed 'safe' by the United Nations were filled with danger, and innocents were abandoned to slaughter. The shame still clings, a generation after the events," Mr Ban said.
How the genocide happened
Commemoration in pictures
Your questions answered
Mr Kagame said Rwanda was "completely broken" after the genocide, but it had managed to unite itself.
"It [the genocide] simply should never have happened," he added.
Other international leaders present include Uganda's President Yoweri Museveni, former South African President Thabo Mbeki and former UK Prime Minister Tony Blair, now an advisor to the Rwandan government.
On Sunday, hundreds of people attended a Mass at Sainte-Famille Catholic church in Kigali to remember those who died in the church itself and elsewhere.
A spokesman for the Rwandan government said France would not be represented at the events to mark the genocide.
Mr Flesch told AFP  that he had received a telephone call from the Rwandan foreign ministry to "inform me that  I was no longer accredited for the ceremonies".
Earlier, the French foreign ministry said Mr Flesch would represent France at the events after Justice Minister Christiane Taubira cancelled plans to attend following an accusation by Mr Kagame that France had participated in the mass killings.
France was a close ally of Mr Habyarimana's government.
In an interview with the French-language weekly news magazine Jeune Afrique, Mr Kagame denounced the "direct role of Belgium and France in the political preparation for the genocide".
He also said that French troops had taken an "active" part in the killings.
France has rejected this charge, with Edouard Balladur, who was French prime minister in 1994, telling Europe 1 radio: "France is in no way complicit in the genocide.
"On the contrary, it of all countries in the world was the only one that took the initiative to organise a  humanitarian operation to prevent widespread massacres."
France did send troops who set up "safe zones" in Rwanda. It says they saved thousands of lives but Rwanda's government has long said they did not do enough to stop the killing - and now says they worked with the Hutu militias carrying out the slaughter.
The UN is still ashamed over its failure to prevent the 1994 genocide in Rwanda, UN chief Ban Ki-moon has said.