Is memcache only for logged in users?
In short, yes memcached will benefit both anonymous and authenticated visitors (even if you also use Varnish).

Memcached and Varnish operate at different layers.  Varnish caches entire HTTP requests (and sometimes edge-side includes, but this is less common in the Drupal ecosystem). Memcached stores frequently used data within the application, and it is a replacement for the cache_* family of database tables.

Memcached will speed up anonymous requests when the Varnish cache misses or is bypassed.