Mr Khan won on second preference votes after failing to gain more than 50% in the first round.
He said: "I'm so proud that Londoners have today chosen hope over fear and unity over division."
The Greens' Sian Berry took third and the Lib Dems' Caroline Pidgeon was fourth. Peter Whittle's fifth place was UKIP's best ever result in London.
The Sadiq Khan story
How Sadiq Khan won
Who would join Sadiq Khan in City Hall?
Mr Khan's success comes after a patchy election night for Labour, who were overtaken by the Conservatives as the second largest party in the Scottish Parliament, fell short of a majority in the Welsh Assembly, but retained control of 57 councils.
Mr Khan gained 44.3% of first preference votes, to Zac Goldsmith's 38.6%. After second preferences came into play, Mr Khan gained a total of 1,310,143 votes, or 56.8%, to the Conservative candidate's 43.2% - making it a more clear-cut contest than in 2012.
It sees Mr Khan become the first Muslim mayor of any capital city in the EU.
Outgoing mayor Boris Johnson said: "Many congratulations to Sadiq on securing a huge mandate to do the best job in British politics. I wish him every possible success."
In a short speech Mr Goldsmith said: "I wish him well as he sets out to build on the success seen under Boris Johnson."
He admitted he was "disappointed, of course, by the result that I won't be able to deliver a manifesto that I'm really proud of".
Mr Khan was congratulated by Labour leader Jeremy Corbyn, who said he would work with the new mayor "to create a London which is fair for all".
Praise also came in from New York mayor Bill de Blasio and Business Secretary Sajid Javid, who tweeted: "From one son of a Pakistani bus driver to another, congratulations".
Labour also saw a boost in their vote across London in the Assembly elections, while several prominent Conservatives criticising their own mayoral candidate's campaign.
Former minister Baroness Warsi called it an "appalling dog whistle campaign" and Ken Clarke said it had been "a mistake" which "probably had a counter-productive effect".
Zac Goldsmith's own sister, Jemima, tweeted: "Zac's campaign did not reflect who I know him to be - an eco-friendly, independent-minded politician with integrity."
Steve Hilton, David Cameron's former director of strategy, told BBC Newsnight  that the Conservatives' approach to the mayoral race had "brought back the 'nasty party' label to the Conservative Party".
He added none of Mr Goldsmith's best qualities were "conveyed in his campaign, which to be honest I found really weird".
But Conservative Assembly Member Gareth Bacon told BBC Radio London that the claims of a negative strategy were "a media-generated fixation".
The election saw the highest-ever turnout at 45.6% - narrowly improving on 2008, when it was 45.3%.
Read more about the London Assembly results.
Sadiq Khan has won the London mayoral election, beating Conservative rival Zac Goldsmith.