Four attackers were seen entering the office, followed by heavy gunfire and several explosions.
Five security officers - including a police chief - were among the dead and 26 government officials among the hurt.
A spokesman for the Taliban said they carried out the attack.
The siege ended seven hours after it began and all four gunmen were killed by the Afghan National Security Forces which had surrounded the area.
Police said they rescued the attorney general from his office, which is just 200m (650ft) from the office of the governor of Balkh province.
The governor's aide told the BBC that the governor, Atta Muhammad Nur, was at home when the attack began but was taken to his office to monitor the situation.
In a statement seen by the BBC, Taliban spokesman Zabihullah Mujahid said: "Breaking News: This noon suicide [attackers] stormed office of the attorney and appeal court of Balkh province. Heavy casualties for enemy, battle continues."
The Taliban has targeted Afghanistan's legal system in the past. In June 2013, the group detonated a car bomb in front of the compound of Afghanistan's Supreme Court, killing 17 and wounding 39.
And in December of 2014, Atiqullah Raufi, the chief of the secretariat of Afghanistan's Supreme Court, was assassinated by Taliban gunmen in the west of Kabul.
The attack in Mazar-e-Sharif comes a day after an American soldier was killed in a fire fight between US and Afghan troops in eastern Afghanistan.
Gunmen dressed in military uniforms have stormed the office of the attorney general in the northern Afghan city of Mazar-e-Sharif, killing at least 10 people and injuring more than 65.