Mr Hollande, whose party was beaten by the far right in last week's European Parliament election, said the EU had become too complex and remote.
In response, he will tell EU leaders at a meeting in Brussels later that they must focus on boosting the economy.
The three big pro-EU centrist blocs are still on course for a majority.
But they have lost seats in the European Parliament to parties seeking to curb EU powers or abolish the union, among them the UK Independence Party which came first in the domestic vote with 27% according to provisional results.
In France, the far-right National Front stormed to victory with a preliminary 25% of the vote, pushing Mr Hollande's Socialists into third place.
National Front President Marine Le Pen said on Tuesday that her party would use its electoral mandate to "defend France"  and fight "crazy measures like votes for immigrants".
The BBC's Matthew Price in Brussels says she made a direct challenge, not just to French politicians, but to Brussels too - saying it must listen to the people, and that the French had to protect their nation.
This agenda will affect policy-making in the EU in the coming years, our correspondent says.
Full coverage of results
Speaking on French TV, Mr Hollande - a leading champion of the EU - said the project had become "remote and incomprehensible", and that that had to change.
"Europe has to be simple, clear, to be effective where it is needed and to withdraw from where it is not necessary," he said.
He said the union had overcome the crisis in the eurozone "but at what price? An austerity that has ended up disheartening the people".
When European Union leaders meet on Tuesday he would "reaffirm that the priority is growth, jobs and investment", he said.
Matthew Price, BBC News, Brussels
Europe has not "voted against the EU". The vast majority of those who bothered to cast a ballot did so for parties that are pro-EU, and they will make up the majority in the new parliament.
Yes, the focus is understandably on France and the UK, with Denmark, Greece, and others also giving Euro-enthusiasts cause for concern. However in many countries mainstream parties dominated - in Germany, Italy, Poland where it was felt a growing Eurosceptic movement could break through, in the Netherlands and elsewhere.
So the results do not constitute a "problem" as such for the leaders of the EU. Many leaders will point this out around the dinner table tonight. Others, however, will stress that a sizeable chunk of voters chose parties that want "Brussels" to change, and that the EU needs to address this issue if it is to maintain popular support and legitimacy in the longer term.
Ahead of this election most leaders were not planning to come to Brussels to discuss how to reset their country's relationship with the EU. David Cameron was of course - but others were not. Will others - under domestic pressure - now join him?
Anti-EU forces overshadow Brussels talks
Mrs Merkel - whose conservative Christian Democratic Union won a comfortable 35% of the vote in Germany - said it was now up to the established parties of Europe to win voters back by focusing on "improving competitiveness, on growth and creating jobs".
"This is the best answer to the disappointed people who voted in a way we didn't wish for," she said.
Elsewhere in Europe, the anti-EU UKIP was celebrating winning 27% of the vote, marking the first time in a century that a party other than the Conservatives or Labour has won any UK election.
Prime Minister David Cameron, whose Conservative party lost seven seats, said it was clear voters were "deeply disillusioned" with Europe and that the message was "received and understood".
But he insisted he would neither bring forward the date of an in/out referendum on UK withdrawal from the EU - scheduled for 2017 - nor seek a pact with UKIP.
Despite the unprecedented Eurosceptic gains across the Union, Jose Manuel Barroso, outgoing president of the European Commission, insisted that the pro-EU blocs still had "a very solid and workable majority".
The centre-right European People's Party appears set to win 213 out of the 751 seats, with 28.36% across the bloc, according to estimated results issued by the European Parliament. That would mean it remains the biggest group - but with more than 60 seats fewer than before.
The Socialist alliance has a projected 190 seats, with 25.3% of the vote, the Liberals 9% and the Greens 7%.
The Eurosceptic Europe of Freedom and Democracy group appeared to have around 38 seats - including the 24 for UKIP. But the number of non-attached right-wing MEPs is set to rise, boosting the Eurosceptic camp.
Mr Barroso said a "truly democratic debate" was needed to address the concerns of those who did not vote, or "voted in protest".
Turnout across Europe is estimated at 43.1%, the first time it has not fallen since the previous election - but it has only increased by 0.1%.
Result highlights (from European Parliament website):
The election is the biggest exercise in multi-national democracy in the world, affecting the lives of the EU's 500 million citizens.
The parliament's powers have expanded since the last election in 2009, and it is hoping to have a decisive say in who gets the EU's top job, president of the European Commission.
You can follow full coverage with all the latest updates at bbc.co.uk/vote2014.
French President Francois Hollande has said the EU must reform and scale back its power, amid a surge in support for Eurosceptic and far-right parties.