"Remove double" is deforming my mesh
When removing doubles from the imported mesh the normals split in a custom way aren't preserved, making some dark patches to appear. Probably some of split normals were saved while others weren't because removing doubles doesn't choose to preserve given vertex just because it stores some custom data.  

In anyway if removing doubles make sure to remove imported normals as well. That will make model lose its look as it was done before exported though.

If the lost shading is important and irretrievable possible solutions can be to manually join double vertices to control which vertex goes where (long way) or recreating shading as it was before removing normals (may be even longer). 

Another way would be to duplicate the object before removing custom split normals, remove doubles,  work with it and once done use Data Transfer modifier to copy normals from one mesh to another.

Adding another layer of custom normals with "Add Custom Split Normals  Data" in the Geometry rollout can eliminate the problem but this solution doesn't seem to be stable as it works somehow differently for different objects.