City centre turns into battle zone as anti-government fighters target several locations.
Mohammad Atmar, the new minister of internal affairs, told journalists that seven of the attackers had been killed.
A ministry spokesman, Zemarai Bashery, added that two of the attackers had taken refuge in a building and had been killed.
Farhad Paiker, an Afghan journalist, told Al Jazeera that a suicide bomber had hit a shopping centre.
"A suicide bomber in a car came towards the foreign ministry. Security forces tried to stop it and it hit a shopping centre," he said.
"It is really chaotic in the area."
Later a car bomb exploded near another shopping centre, close to the education ministry.
A security source was quoted by the Reuters news agency as saying that a suicide bomber had killed "several police and intelligence officials".
Qais Azimy, Al Jazeera's Kabul producer, reporting about 200m from the scene of some of the fighting, said: "There are hundreds of Afghan army and police and intelligence officials present. Civilians have completely left the area.
"It is a big question mark how they [the fighters] got so close to the presidential palace."
David Chater, Al Jazeera's correspondent who was at the Serena Hotel around which some of the heaviest fighting took place, said: "We've heard four large explosions very close to the hotel."
"It is extraordinary that security has been breached to this extent," he said, adding that it showed the Taliban could act at will.
The Serena Hotel, which is frequented by foreign journalists, was reported to be on fire shortly after the attacks began.
Chater said there had been recent warnings that opposition fighters had hijacked six armoured cars and were planning an attack on the capital.
The attack appears to be the most co-ordinated offensive on the capital since the US-led invasion in 2001 that toppled the Taliban from power.
Afghan politicians insisted the Taliban attack held little significance.
"It does not mean the Taliban is strong. It means they cannot target military targets only the Afghan people. It means that they are weak," Mir Ahmed Joyenda, an Afghan parliamentarian, told Al Jazeera.
Even so, the attack comes at a sensitive time in Afghan politics with Karzai yet to finalise his cabinet after disputed elections.
Chater said that the strikes will bring into question the credibility of Karzai's authority and the military strategy in Afghanistan of Barack Obama, the US president.
The attack also comes ahead of the London Conference on Afghanistan hosted by the UK, UN and Afghanistan on January 28 on winning the conflict in the country.