Varnish doesn't cache first hit at all, only second hit I have Varnish 4 with Drupal 7. In Drupal settings.php I have:

// Tell Drupal it's behind a proxy.
$conf['reverse_proxy'] = TRUE;

// Tell Drupal what addresses the proxy server(s) use.
$conf['reverse_proxy_addresses'] = array('127.0.0.1');

// Bypass Drupal bootstrap for anonymous users so that 
// Drupal sets max-age &amp;lt; 0.
$conf['page_cache_invoke_hooks'] = FALSE;

// Make sure that page cache is enabled.
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
// Set page cache max-age to 600 secs (10 min) 
// Change this to the highest time possible for your needs
// Drupal will set a Cache-Control: "max-age=600" header 
$conf['page_cache_maximum_age'] = 21600;

$conf['cache_backends'] = array('sites/all/modules/varnish/varnish.cache.inc');

$conf['cache_class_cache_page'] = 'VarnishCache';


In the varnish vcl file I have:

vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.

 ## ADMIN PAGES ##

    # Here we filter out all URLs containing Drupal administrative sections
    if (req.url ~ "^/status\.php$" ||
        req.url ~ "^/update\.php$" ||
        req.url ~ "^/admin$" ||
        req.url ~ "^/admin/.*$" ||
        req.url ~ "^/user$" ||
        req.url ~ "^/user/.*$" ||
        req.url ~ "^/flag/.*$" ||
        req.url ~ "^.*/ajax/.*$" ||
        req.url ~ "^.*/ahah/.*$") {
           return (pass);
    }
}
sub vcl_backend_response {
    # Happens after we have read the response headers from the backend.
    #
    # Here you clean the response headers, removing silly Set-Cookie headers
    # and other mistakes your backend does.
if (bereq.url ~ "(?i)\.(css|js|jpg|jpeg|gif|png|ico)(\?.*)?$") {
        unset beresp.http.set-cookie;
    }
# Set cached objects to expire after 1 hour instead of the default 120 seconds.
    set beresp.ttl = 4h;
}

sub vcl_deliver {
    # Happens when we have all the pieces we need, and are about to send the
    # response to the client.
    #
    # You can do accounting or modifying the final object here.
if (obj.hits &gt; 0) {
               set resp.http.X-Cache = "HIT";
        } else {
               set resp.http.X-Cache = "MISS";
        }
}


This is my header at isvarnishworking:

HTTP/1.1 200 OK
Date:   Fri, 02 Mar 2018 13:51:24 GMT
Content-Type:   text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=dfac4e840061f192552b77dfd3430fcfc1519998683; expires=Sat, 02-Mar-19 13:51:23 GMT; path=/; domain=.mydomain.net; HttpOnly
X-Powered-By:   PHP/5.4.45
X-Drupal-Cache: MISS
Expires:    Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control:  max-age=64800, public
X-Content-Type-Options: nosniff
Content-Language:   en
X-Frame-Options:    SAMEORIGIN
X-Generator:    Drupal 7 (http://drupal.org)
Vary:   Accept-Encoding,User-Agent
X-Varnish:  459087
Age:    0
Via:    1.1 varnish-v4
X-Cache:    MISS
Server: cloudflare
CF-RAY: 3f54567d343346ec-EWR
Content-Encoding:   gzip


I'm using Cloudflare, but I've already been try to shutdown Cloudflare for 2-3 days. Still, Varnish doesn't hit the first time, only the second time.
It looks like a normal behaviour : 


first url hit : Varnish puts the page in cache with some id
only following hits will serve the cached id instead of querying the backend. 


If you want anonymous users to be always served by varnish, you need to implement a cache warmer logic.

Setup a sitemap. Then create a cron task that will parse the sitemap with curl for instance,  and run that cron task at a frequency synced with your cache expiry time...