Setting track points from Mocha to a solid makes it very small
Unbelievable,

Right after I asked the question I found the solution online.
After tracking in Mocha make sure you click the "Align selected surfaces" button.
I had to search for it but here it is:


After this you can copy your tracking data and paste it in AE.