The EU is expected to renew sanctions in a matter of weeks.
Alexis Tsipras was speaking at a joint news conference in Athens with the Russian President Vladimir Putin.
Mr Putin said there would be "no discussions" about Crimea, the Ukrainian peninsula that was seized by Kremlin-backed forces in 2014, leading to sanctions from the EU and US.
The territory, which has an ethnic Russian majority, later voted to join Russia in a referendum that Ukraine and Western countries deem illegal.
How Russia's relationship with Europe has evolved
Mr Tsipras told reporters: "We have repeatedly said that the vicious circle of militarisation, of Cold War rhetoric and of sanctions is not productive. The solution is dialogue."
Mr Putin said: "As far as Crimea is concerned, we consider this question is closed forever. Russia will not conduct any discussions with anyone on this subject."
Mr Putin's visit to Greece is an attempt to reinforce a relationship with one of Russia's friends in Europe at a time when diplomatic tensions continue between Russia and the EU and US, the BBC's Thomas Fessy in Athens says.
Mr Putin also said that Russia would be forced to respond to US moves in Europe, warning that Washington's missile shield bases in Romania and Poland were a direct threat to his country's security.
"If yesterday in those areas of Romania people simply did not know what it means to be in the cross-hairs, then today we will be forced to carry out certain measures to ensure our security."
Earlier this month, the US activated the anti-missile base in southern Romania. The base in Poland is expected to be operational in 2018.
The US says its shield is to protect Nato countries from short- and medium-range missiles, particularly from the Middle East.
The Greek prime minister has said sanctions imposed on Russia over its actions in Ukraine are not productive.