Some templates are not properly compiled to Twig
OK!
So I have figured it out. I think this is a Windows-specific bug because, on another project where I have experienced this as well, people with macs do not have this issue.

Anyway, what is causing it is \Drupal\Component\PhpStorage\MTimeProtectedFastFileStorage::getFullPath, specifically return $directory . '/' . hash_hmac('sha256', $name, $this-&gt;secret . $directory_mtime) . '.php';

If the template name is too long, which can easily happen with theme suggestions, it will cause PHP to fail to perform the rename() operation successfully. I don't know why it stores the code in a temporary file and then it copies it into target directory instead of creating it directly in the target directory and/or file but that's how it is. 

I have no idea why we need so freaking long directory and file names when they only hold a single file but beside this, Drupal 8 is great.