Views entity object sometimes missing
This is because I have search specific caching enabled on the view. This is a Search API caching plugin for views which caches both the results of a query and the HTML (I think).

So when a query is repeated it returns the HTML and doesn't load the entity objects. Because that's the entire point of the cache! (In fact it skips the entire view generation, i.e. both hooks like module_views_post_render and theme_preprocess_views_view)