Thanet Council began work two weeks ago at All Saints Church in Birchington but some stones have been left broken.
One resident, Janice Taylor, said it looked as if "hooligans and demolished the place".
Thanet Council admitted the work could be "upsetting" but said it was in line with national guidelines.
These are determined by the Ministry of Justice and its Burial and Cemeteries Advisory Group.
The work also has the approval of the vicar of Birchington, the Rev Done Witts, the authority said.
Ms Taylor said she was appalled by how the work had been left.
"It looks like an act of vandalism has occurred.
"It doesn't look as if the council has come in here to do work, it looks as if hooligans have come in and demolished the place."
Mr Witts said: "It's just that it looks untidy in the meantime until the council sorts it out."
The council said there were more than 100,000 memorials across cemeteries in Thanet, many of which were more than 100 years old.
It said the testing was being done to ensure the gravestones did not pose a danger to the public.
"The council appreciates the sensitivity of this work with every effort made to advise those affected," it said in a statement.
"It has produced a question and answer leaflet to guide plot owners through what could be a potentially upsetting time."
It also said a team at Thanet Crematorium were available to advise on "all aspects of the Memorial Stability Programme".
A Kent council has been accused of leaving gravestones and memorials looking as if they have been vandalised after carrying out safety checks.