How to UV Smart-Unwrap Properly?
I have found a Solution. This is not really an answer as to why it is happening but it's too long to put into a comment and at least it fixes the problem.


Go into Edit Mode
Press P (for Seperate) and select By Loose Parts
Create a Cube somewhere in the scene
Select all the loose parts of your model and then the cube so the
cube is the active object and press Ctrl+J (for join)
Go back into Edit mode and delete the Cube, unwrap: done.


This is the same procedure by the way that I frequently use to set the rotation and scale of an object to 1.