The Online Hate Crime Hub is made up of five specially trained Met police officers who will try to identify, prevent and investigate online abuse.
Sadiq Khan said officers would "work with community experts to develop the police's understanding of online hate".
It is the first hub of its kind in the UK and will cost Â£1.7m over two years.
It is being funded by the Met and the Mayor's Office for Policing And Crime (MOPAC), with Â£452,000 also being contributed by the Home Office Police Innovation Fund.
Any online hate crimes reported to police in the capital, including abuse posted on Twitter and Facebook, will be looked into by the unit.
Officers will provide referrals to victim support groups and work with police in relevant boroughs to investigate.
City Hall said discussions were also under way with social media companies "to develop appropriate online sanctions for perpetrators of online hate".
Victoria Wright, a disability and disfigurement rights campaigner in London, who has been subjected to online abuse, called the hub "a much needed initiative that will make a real difference".
"It's vital that those of us who are victims of online hate crime receive a robust response from the police," she said.
A police unit to help tackle online hate crime and provide better support for victims in London has been launched by the capital's mayor.