It said it plans to replace them with less obtrusive surveillance.
However Muslim leaders have called on worshippers to continue boycotting the sacred compound for now.
There were deadly clashes after the metal detectors were set up, which Palestinians saw as an Israeli attempt to assert control over the site.
Israel said they were necessary to prevent weapons being smuggled in.
It followed the killing on 14 July of two Israeli policemen by Israeli-Arab gunmen, who police say had hidden their weapons on the hilltop site known to Jews as the Temple Mount and Muslims as Haram al-Sharif.
The Israeli prime minister's office said the state security bodies had recommended replacing the metal detectors with "advanced technologies and other means".
It said the new, unspecified, measures, costing 100m shekels ($28m; Â£21m), would be put in place over the next six months. Until then, extra police would be deployed around the site, it added.
However, the Muslim body which oversees the holy compound, the Waqf, called on worshippers to stay away from the site until it advised otherwise.
Many Palestinians had been refusing to visit the complex as long as the metal detectors were there, holding prayers in the street outside the Old City instead.
Tom Bateman, BBC Jerusalem correspondent
The crisis at this revered site was about much more than the existence of the airport-style gates and electronics of the metal detectors themselves.
For Palestinians, their installation symbolised what they saw as an attempt by Israel to assert further control over the site of their holy shrine in occupied East Jerusalem - the mosque that also acts as a powerful symbol of their national aspiration.
For Israel's government, the move represented the need to secure what one minister called "the most sensitive location on Earth" - Judaism's most holy place - after the murder of two Israeli police officers close by.
Leaders will have felt pressure from within their own constituencies not to be seen to back down.
International calls for a solution have added to the pressure on both sides - which were dealing with complex internal politics too - but it is still not clear this crisis has been entirely resolved.
The installation of the metal detectors the day after the police were killed drew angry protests from Palestinians and days of clashes with Israeli security forces.
The violence left four Palestinians dead in East Jerusalem and the occupied West Bank, while three Israeli civilians were stabbed to death by a Palestinian who had said he was avenging Israeli measures at the holy site.
On Monday the UN's Middle East envoy warned of "potential catastrophic costs" even beyond the region if the issue was not resolved by Friday, when Muslims traditionally flock to the site for prayers.
There is speculation that Israel agreed to remove the metal detectors in return for Jordan letting Israeli embassy staff leave the country following a diplomatic stand-off.
Jordanian authorities wanted to question an Israeli security guard who Israel said shot dead a Jordanian who had attacked him with a screwdriver in a residence next to the embassy. A second Jordanian who died was inadvertently killed by the gunfire, Israel said.
Israel said the guard had diplomatic immunity and both he and all the staff were confined to the embassy compound amid attempts to find a solution.
In a phone call on Monday night, Jordan's King Abdullah and Mr Netanyahu discussed the crisis surrounding the holy site and the incident at the embassy. The king called on Mr Netanyahu to remove the metal detectors. At around the same time, it was announced that the embassy staff had returned to Israel.
Jordan, which has a large Palestinian population, has an historical role as custodian of the Jerusalem holy site.
Mr Netanyahu's office said the embassy staff's return "was made possible thanks to... tight co-operation" with Jordan.
The Temple Mount/Haram al-Sharif in Jerusalem's Old City is sacred to both Jews and Muslims. Jews revere it as the location of two Biblical Temples and holiest site in Judaism. It is also the al-Aqsa mosque compound, the third holiest site in Islam.
The area, in East Jerusalem, has been under Israeli occupation since the 1967 Middle East war.
The site is hugely politically sensitive and has been subject to a delicate set of arrangements - commonly referred to as the "status quo" - governing access, security and administration, for the past 50 years.
Under the arrangement, Israel is responsible for security and a Jordanian-funded religious trust, or Waqf, looks after the day-to-day running of the site.
Palestinians said the placing of the metal detectors upset the status quo, which Israel has repeatedly pledged to maintain. Israel said Palestinians were using the issue as a pretext to spread hostility against the Jewish state.
Israel has removed metal detectors from outside a holy site in East Jerusalem after uproar from Palestinians over their recent introduction.