The Yemeni troops, along with a force of local tribesman, were backed by the United Arab Emirates and US forces.
The US views Al-Qaeda in the Arabian Peninsula (AQAP) as the most dangerous wing of the terror group.
AQAP has taken advantage of the chaos of the Yemeni civil war to expand in the oil and gas rich south.
The aim of the mission was to "degrade" AQAP's abilities, Pentagon spokesman Navy Captain Jeff Davis said.
But local people said the militants did not put up a fight, and simply melted away into the mountains.
As a result, the province's main cities are said to be under government control for the first time in years.
Forces backing Yemen's government have reportedly driven al-Qaeda fighters out of the main cities of the southern province of Shabwa.