Removing superfluous vertices in a large mesh
I have tried to mimic your scene and this worked quite well for me... so maybe it is worth to try:

EDIT: It looks like this method works only for a simple model like in my example, if it is more complex scene, you need to divide it into smaller parts otherwise the Remesh modifier does not reconstruct the model properly.


Join all blocks into one mesh
Add Remesh modifier, untick Remove Disconnected Pieces and play with the Octree Depth to get the initial shapes
Add Decimate modifier - planar
Apply modifiers




Then just apply that last modifier (the gif was too big to upload so I had to cut it there).

The result should look like this: