Opposition councillors accused the ruling Labour group of "arrogance" and not listening to the public's concerns.
The plans had also prompted protests outside the council's offices on Wednesday.
Council leader Peter Marland said he was "disappointed" but would work with other parties to agree a new budget.
The authority's provisional 2015-16 budget had included £1.5m cuts to bus services, and reduced funding for the Citizens' Advice Bureau.
Homeless shelters faced possible closure under the plans, and the removal of other funding could have forced some scout groups to shut down.
Liberal Democrat leader, Douglas McCall, accused council leaders of refusing to listen to public concerns or engage with opposition parties.
He described their approach to a public consultation as "arrogant".
However, Mr Marland said a record consultation response showed the authority was listening.
"What we couldn't do is persuade enough opposition politicians that what we were doing was ultimately right," he said.
Former Conservative group leader Andrew Geary said his party had predicted savings for five years and put forward a "clear strategy" for delivering them when it was in control.
"You simply can't amend something that is not built on a solid foundation and this budget is built on the sand and it will collapse and it won't deliver."
The authority is set to meet again next week to discuss the revised proposals.
Analysis
Paul Scoins, Political reporter, BBC Three Counties
After the political grandstanding comes the negotiation.
Concessions will be made but the key sticking point of provision for homeless people will remain a subject for debate.
The council will then need to agree that budget and, if they fail, the government will step in and act, most likely freezing the budget in its current state.
More than £90m needs to be saved over the next few years and Labour says difficult decisions are inevitable.
Sources within the leadership group say they would be surprised if an agreement cannot be reached, suggesting the "nuclear option" of handing the decision to communities secretary Eric Pickles would be something all parties would find difficult to swallow.
Proposals for £22m council budget cuts in Milton Keynes have been rejected over fears they would adversely affect vulnerable people.