In Bucharest some protesters broke through police lines and headed towards government buildings.
Prime Minister Sorin Grindeanu says the move would ease overcrowding in prisons.
His critics say he is trying to release allies convicted of corruption.
They also want the proposals to be debated in parliament.
Mr Grindeanu's plan is for the measures to be implemented through emergency decrees that bypass parliament and do not require the signature of President Klaus Iohannis.
Those who would be due for release include several elected officials and magistrates convicted of corruption.
President Iohannis joined protesters in Bucharest on Sunday, saying they were rightly indignant.
"Several political officials who have judicial issues want to change the legislation and weaken the rule of law," he said.
"It is unacceptable to modify the law so that the cases of dozens, even hundreds, of politicians, are wiped out."
President Iohannis was elected in November 2014 and has vowed to stamp out entrenched corruption.
However, the chairman of the ruling Social Democrats, Liviu Dragnea, accused the president of "inciting social disorder and violence" and said the protest was the start of a coup.
A former minister, Mr Dragnea was himself given a suspended prison sentence in 2015 for using bribes and forged ballot papers to influence an impeachment vote in 2012 against then President Traian Basescu.
Tens of thousands of people have taken to the streets of the Romanian capital Bucharest and other cities in protest at government plans to release around 3,000 prisoners.