How to enable Views caching for all pages instead of only the first page? Problem

I'm trying to cache my view (search specific). However doing so only caches a single page of results and stops me from changing pages either with the pager, or by manually changing the URL.

Has anyone else run into this problem? And if so how have you fixed it?

Details &amp; Thoughts

The hyperlinks on the pager still look ok. It's just that using them only gives the first page of results so it seems to do nothing.

It seems like Drupal is only capable of caching a single page of results and then returns that for all queries. 

For example: If I change the caching (signalling the cache needs to be rebuilt) then enter the URL for page 3 of views results, then it caches that instead of the first page and will only return that for all searches.

My Set-Up is:


View Based On: Search API index. 
Caching: Search Specific | 1 Hour


And caching mechanism's I have running are:


Memcached 
Authcache  
Varnish


I don't think it's related to Authcache. If I turn it off I get the same problem. (I presume view caching is separate and un-related to caching based on users. Also see the next point.)

I'm running Authcache Debug and it's telling me that it can't cache the view because of a client form token. (I have an exposed form on the views page which will block Authcache.)
While I haven't used a search view, I ran into a similar problem with pagers. Views caching has two parts out of the box, query and render. The render cache was interfering with the pager. I think setting the Views caching to Search Specific is setting both query and render in one swoop. 

I would try changing it to Time Based and caching the query, but not the render. Varnish will then be able to correctly cache the various pages of the view.