Eagle clearance issue
I have solved the problem. I will state below what the solution is so that others can learn from it:


I started drawing the package with the dimension layer.. I know, don't do that.
Then I replaced the layer with the silkscreen 21 tplace layer. This is the layer that should be used if the component outline should be visible on the final PCB.
What I should have done is updating my library. I thought I did this, but probably failed to double check. 


Now everything works fine. Thanks for the support.