About 2,000 pieces dating back more than 1,000 years were spotted on the seabed by members of a diving club, the Israeli Antiquities Authority said.
At first the group thought the coins were toys, but quickly realised the significance of their discovery.
The Antiquities Authority said the find was "so valuable that it's priceless".
Divers were exploring an ancient harbour in Caesarea when they came across the gold. They collected several coins and quickly returned to the shore to inform the director of the club.
Experts from the authority called to the site uncovered "almost 2,000 gold coins in different denominations" circulated by the Fatimid Caliphate, which ruled much of the Middle East and North Africa from 909 to 1171.
Kobi Sharvit, director of the marine archaeology unit at the Israel Antiquities Authority, said excavations would be carried out in the hope of shedding more light on the origin of the treasure.
He added: "There is probably a shipwreck there of an official treasury boat which was on its way to the central government in Egypt with taxes that had been collected.
"Perhaps the treasure of coins was meant to pay the salaries of the Fatimid military garrison which was stationed in Caesarea and protected the city."
He said another theory was that the treasure belonged to a large merchant ship that traded with the coastal cities and the port on the Mediterranean Sea.
The haul, which weighs 9kg (20lb), is believed to have been exposed as a result of winter storms.
It is now the property of the state and there was no finders' fee.
Scuba divers have inadvertently discovered the largest trove of gold coins ever found off Israel's Mediterranean coast.