The vital vote focused on the proposed £140m plan - backed by £50m from businessman Sir Ian Wood - to transform the historic Union Terrace Gardens.
In a referendum earlier this year, there were 45,301 votes in favour with 41,175 votes against.
New proposals from the Independent Alliance Group were backed by 22, with 20 against and one abstention.
Improvements to Union Terrace Gardens, and landmark buildings, are instead proposed.
There were heated exchanged afterwards between some business leaders who supported the City Garden Project and councillors who voted against it.
Sir Ian's £50m offer has now been withdrawn. He said he was dismayed.
Callum McCaig, leader of SNP group, had called for the City Garden Project to be backed.
He said he was "bitterly disappointed" at the decision.
Conservative Alan Donnelly said the amendment was an "almighty fudge".
However, Mike Shepherd, chairman of Friends of Union Terrace Gardens, said: "There is no doubt that our councillors made the correct decision.
"This is the end of the City Garden Project debacle.
"Union Terrace Gardens is at the heart of our city. Let's nurture them with the care and attention they deserve. We can restore them to a place of pride and glory. Let's go ahead and do it."
The abstention was from Liberal Democrat Ian Yuill.
Various delegations had addressed the full meeting of Aberdeen City Council during the day.
One of the first to speak among earlier delegations was Leo Koot, managing director of oil firm TAQA Bratani, who said Union Terrace Gardens was a "dark hole".
However, that claim was contested by Iain Richardson, from Common Good Aberdeen.
Prominent Aberdeen hotelier Stewart Spence said the city needed to attract tourists, and required more than a spruce up.
Dan McCroskrie, speaking on behalf of the young people who support the scheme, said it was a once-in-a-lifetime opportunity which would help attract more young people into the city.
The cost of the winning Granite Web design was £140m.
The council administration's senior coalition partner, Labour, had vowed to scrap the scheme.
Aberdeen's controversial City Garden Project has been narrowly rejected after a council debate.