Benjamin Netanyahu's statement came after an emergency meeting of his cabinet and security chiefs.
Alexander Levlovitz died in a car accident apparently caused by a rock-throwing attack in Jerusalem.
Meanwhile, clashes between Israeli police and Palestinian protesters continued for a third day in the city.
"On the eve of the Jewish new year, we once again saw how throwing stones can kill," Mr Netanyahu said in his statement.
"These kinds of activities will be met with a very fierce punitive and deterring response."
The government agreed to establish mandatory minimum penalties for those who "endanger human lives by throwing stones, fire-bombs and explosives".
It also said it would introduce heavy fines on parents who allow their children to engage in violent rioting.
Mr Levlovitz died and two passengers were reportedly injured after their car was pelted with stones on Monday. Police are investigating the incident.
Separately, violence has again rocked the al-Aqsa mosque compound.
The compound - known to Jews as the Temple Mount and to Muslims as al-Haram al-Sharif - is the holiest site in Judaism, and contains the al-Aqsa Mosque - the third holiest site in Islam.
The compound is a source of religious and political tension between Israel and the Palestinians. It is a frequent flashpoint for violence.
On Tuesday, police spokeswoman Luba Samri told AP that police entered the Al-Aqsa mosque compound early in the morning to disperse protesters who had stayed inside the mosque overnight.
The protesters threw projectiles at officers, Ms Samri said, adding that two Palestinians were arrested and five police officers were slightly injured.
Twenty-six Palestinians were injured on Tuesday, none of them seriously, the director of the Palestinian Red Crescent emergency unit, Amin Abu Ghazaleh, told Reuters.
UN Special Envoy Nickolay Mladenov later warned that unrest in Jerusalem "may ignite violence beyond its walls".
He also urged all leaders "to ensure visitors and worshippers demonstrate restraint and respect" at holy sites.
The Israeli prime minister has vowed to "use all necessary means" to stop stone throwers after an Israeli man died in a car crash linked to such an attack.