hook_page_alter called once
Theme settings are specific to each theme. Go to /admin/appearance/settings/yourtheme and check for any theme-specific caching options. I can't tell you exactly what the options would be because that depends on your theme.