index.php is added in front of paths for anonymous users
This is mainly an SEO issue. Search engines seem to like unclean URLs with index.php. Once they found one all internal links on your site also contain index.php and so they sometimes crawl the entire site in this namespace.

You have to return a 301 redirect for any unclean URL to stop this. Install the module Redirect and go to the module settings /admin/config/search/redirect/settings:


  GLOBAL REDIRECTS
  
  [x] Enforce clean and canonical URLs.
  
  Enabling this will automatically redirect to the canonical URL of any page. That includes redirecting to an alias if existing, removing
  trailing slashes, ensure the language prefix is set and similar
  clean-up.