Deleting voxel artifacts (averaging small geometry)
Not an answer ...

v01_Vertex Group
I wanted to isolate mess on edge by curvature with Pointiness node (only for Cycles), Bake ...

... and use this texture for VertexWeightProximity modifier to convert into vertex group to be used as mask for Smooth modifier.


v02_Sculpt Mask
Similar technique can be achieve by Sculpt mode. Expand by Curvature didn't work for this, but in the same menu is Dirty Mask that worked. So probably easier way to isolate ...


Smoothing
... but even separation was successful, using Smooth modifier (for vertex group) or Smooth Filter (for sculpt mask), it failed at the end for the same reason ... topology is too bad to get rid of those things.
Even I tried Remesh &gt; Blocks to get even distribution without small artefacts and than Smooth modifier (or any of a smooth modifier).
Sorry
So, probably only one advice I how - try Meshlab app that is announcing nice optimisation features specifically for thouse issues. But I don't have an experience to navigate you more.