Cannot add 'Page Title' and 'Main Content' blocks to panel
Answering my own question after I found some more info. 

This is indeed a bug, hopefully to be fixed soon: https://www.drupal.org/node/2626090