Installation profile glitch
Those are not bugs and have nothing to do with locale.

The standard install profile sets up the administrator role, the theme configuration and many other things, see standard_install() and its default configuration files like the overriden system.theme.yml.

I guess you mean the local tasks with "middle part of the GUI", that is a block and also set up by Standard, like many other blocks too.

If you create a new theme, it makes sense to look at standard on how it does things and copy the parts you need from it.