What is the best way to use Panels
I found this article:

http://ykyuen.wordpress.com/2011/09/13/drupal-customized-user-registrationloginforgot-password-page-with-panel/

Which describes how to workaround this issue with Panels.

This worked well for me, didnt take long at all, but it seems a little hackish. If anybody knows a better way please let me know.