Mr Gustafsson, 42, is being flown back to Sweden from Africa, Sweden's Foreign Minister Margot Wallström said in a statement on Monday.
He was seized by al-Qaeda in the Islamic Maghreb (AQIM) along with two other men, one of whom was freed in a dawn raid in 2015.
Ms Wallström said Mr Gustafsson was "in good spirits", local media report.
"It is with great pleasure that I can announce that Johan Gustafsson has been released," Ms Wallström added.
She said that the Swede's release was thanks to "extensive efforts" and co-operation between the Swedish foreign ministry, police and "foreign authorities".
Ms Wallström said she had spoken with Mr Gustafsson, who she described as being "happy" and "overwhelmed" by Monday's events.
"I cannot say more at the current time," she added.
Sweden's former Foreign Minister Carl Bildt said that Mr Gustafsson's kidnapping weighed heavily on his mind during his time in the role, which ended in 2014.
He tweeted on Monday: "Extremely gratifying that Johan Gustafsson is free. No single case concerned me more as foreign minister."
Mr Gustafsson was kidnapped along with South African Stephen McGown and Dutchman Sjaak Rijke.
Mr Rijke was freed by French special forces in April 2015 after he was discovered by chance in a dawn raid in northern Mali.
AQIM took a number of Western hostages before the French military deployed its forces in January 2013.
In a separate incident in December 2014, French hostage Serge Lazarevic was freed after a prisoner swap.
Hostage Johan Gustafsson, held by al-Qaeda in Mali since 2011, has been freed, the Swedish government says.