He faced criticism online after comparing costume designer Jenny Beavan to a "bag lady" when she picked up her Bafta for Mad Max: Fury Road.
The presenter later insisted Beavan was a "dear friend" and had "got" the joke.
Fry said he has not "slammed the door" on Twitter but said the site had become a "stalking ground for the sanctimoniously self-righteous".
Writing on his official website, he said people needed to "grieve" for what Twitter had become, writing: "I like to believe I haven't slammed the door, much less stalked off in a huff throwing my toys out of the pram as I go or however one should phrase it.
"It's quite simple really: the room had started to smell. Really quite bad."
Fry has been presenting the Bafta film awards for 11 years and audiences have become used to his cutting wit, often involving quips about many of the stars involved.
Beavan, who won the Bafta for Best Costume Design for Mad Max: Fury Road, came onto the stage at London's Royal Opera House wearing a black leather jacket, white t-shirt and dark trousers.
Following her acceptance speech and once she had left the stage, Fry said:  "Only one of the great cinematic costume designers would come to the awards dressed like a bag lady."
But he posted a picture of the pair at a party later in the night to show his comment had not been taken badly, captioning it: "Jenny Baglady Beavan and Stephen Outrageous Misogynist Swine Fry at the after party."
Fry followed that up by saying in an expletive-ridden tweet that his critics were "tragic people".
His latest comment continued: "A stalking ground for the sanctimoniously self-righteous who love to second-guess, to leap to conclusions and be offended - worse, to be offended on behalf of others they do not even know.
"It's as nasty and unwholesome a characteristic as can be imagined. It doesn't matter whether they think they're defending women, men, transgender people, Muslims, humanists â€¦ the ghastliness is absolutely the same."
He said "the tipping point had been reached" but added: "I don't feel anything today other than massive relief, like a boulder rolling off my chest. I am free, free at last."
The former QI host had more than 12 million followers on Twitter at the time he left the site.
Glamour editor Jo Elvin was among those who criticised Fry for his "uncool" comments, saying she was glad Beavan "wore what she wanted".
Comedian Tiernan Douieb described Fry as looking like he was dressed like "an unused Subbuteto (sic) table".
Showbiz journalist Beth Neil wrote: "Stephen Fry's 'dressed as a bag lady' joke about the winner of costume design was horrid. True colours, there."
Fry received support from Twitter users however, including several celebrities.
Radio 1 Scott Mills DJ wrote: "I thought Stephen Fry was an excellent host tonight. Dealing with the biggest stars in the world is scary. I find this. He is a natural."
Comedian Matt Lucas joked: "Stephen Fry Didn't you get the memo? No-one is allowed to do jokes anymore."
British singer Adele - who has 25m followers - announced she was leaving Twitter in 2012 due to comments about her new-born son. She since returned, but revealed last year that she no longer sends her own messages due to "drunk tweeting".
Girls creator Lena Dunham said she was leaving Twitter last year to create a "safe space", after receiving online abuse for posting a picture on Instagram in a sports bra. Although her account is still active, it's believed to be run by a member of her creative team.
Actor and writer Matt Lucas left Twitter four years ago after a joke was tweeted about the death of his former civil partner, Kevin McGee. He has since rejoined the site.
US singer Chris Brown deleted his account in 2012 after receiving a barrage of tweets following a heated exchange with comedian Jenny Johnson. He was back on the site less than a week later.
In 2012, Kanye West quit the site by deleting all his tweets except for one announcing the release date of his Yeezus album and the words: "Be Back Soon." True to his word he was back, after a short break.
Fry left the social media site from February to May 2015 without giving reasons. He also pleaded for fans to stop sending him direct messages saying "it was impossible" for him to look at them if he wanted to have "anything close to a life."
In 2014, Fry quit the site saying it was "unsafe" for him to tweet, adding he was filming "in a place whence I've been advised it is safest not to tweet".
And in 2009, he said he was going to leave after another user called his posts "boring".
Bafta show host Stephen Fry has confirmed he has left Twitter declaring "the fun is over".