You normally see the Roma woman with the colourful headscarf and a Costa coffee cup looking for change outside the Book Centre on John Roberts Square in Waterford.

Then there's the middle- aged man who often sits at the back entrance to the City Square shopping centre. And sometimes there's the teenager near the Bank of Ireland on the quay with a cup in his hand. Yesterday they had disappeared.

The city's small Roma community is living in fear, say its members, and are afraid to show their faces in public.

Almost 200 people gathered in Waterford city last Saturday evening outside the home of a large Roma family – including toddlers and children – and staged a protest in which windows were smashed and doors broken down. Some Roma families had to be evacuated from their homes and are now living at undisclosed addresses.

The following night a smaller group also protested and kicked at the doors of a building they believed housed members of the Roma community.

It was the culmination of tension which has been building for some time – most of it whipped up over social media – over street begging and crime in the city. Numerous Facebook pages have appeared over recent months, labelling members of the Roma community as scum and threatening to "burn the cockroaches out". Most have since been taken down, only to be replaced with new pages.

A counter-demonstration in the city last night, organised by Waterford Against Racism, called for calm and opposed any attempt to tackle local problems through vigilantism. About 100 people gathered on a drizzly night in John Roberts Square, against the backdrop of a heavy Garda presence.

"Yes ,there is crime in Waterford and yes, there is crime in the Roma community," Seamus O'Brien, a member of Waterford Against Racism, told the gathering. "But there's also crime in my family and yours. We can't allow ourselves to descend to mob rule and racist commentary on social media to solve this. We need to say 'no' to racism, and 'yes' to ending poverty."

He said the Garda superintendent had told local politicians and representatives that the vast majority of crime in the city had nothing to do with the Roma community.

'People are very, very scared'

Stelian Ciuciu, of the Roma Integration Association, said many members of the community had little choice but to beg, and pleaded for greater understanding. "People are very, very scared . . . These are people who are not educated. They do not get social welfare, many of them. This is what they know. People need to understand this," he said.

Few elected representatives from the major political parties were present at the gathering, although all parties have strongly criticised last weekend's trouble.

Across the city in Manor Street, residents who agreed to be interviewed had mixed feelings over the local disturbances at the weekend.

None defended the attacks on the local homes; and they claimed local criminal elements were involved in the intimidation of the Roma. But many felt issues involving crime and begging needed to be addressed by the Garda.

"I pride myself on not being a racist," said Bernard Cleary (39). "I try not to label people. But there are issues that need to be addressed. But the begging is an issue and I know people who've been victims of crime involving the community."

Another man in his 60s – a former Waterford Crystal worker – said the begging was tarnishing the image of the city at a time when it was trying to attract tourists.

"A lot of people are sick of it. We're doing our best to attract these people. If it keeps up, they just won't come any more."

The attacks at the weekend have also taken on a political edge, with some politicians privately blaming others of stoking up anger.

Sinn Féin councillor John Hearne, who has taken a strong stance against crime in the city, has previously highlighted crime involving some foreign elements. But yesterday he insisted any targeting of the Roma community was reprehensible.

Most in the city agreed the actions of just 200 people last weekend did not represent the view of the vast majority of the city's 50,000 residents.

"Begging is a problem," said local resident Dermot Keyes. "The problem here is we don't have enough gardaí to tackle street crime. Some Roma might be involved but by far and away the biggest crime problem we have is home-grown and involves people born and bred in the city."