Does DRUPAL_CACHE_PER_PAGE count pager pages (i.e. frontpage?page=3)? I guess my whole question is pretty much in the title... I'm wondering if blocks that use  DRUPAL_CACHE_PER_PAGE get cached on pager pages (i.e. mysite.com/frontpage?page=3), or if the pager query is disregarded (so the block would only be cached once for frontpage, frontpage?page=2, frontpage?page=3, etc.)?
Yes.

See drupal_page_get_cache() in bootstrap.inc:

if (drupal_page_is_cacheable()) {
  $cache = cache_get($base_root . request_uri(), 'cache_page');
  if ($cache !== FALSE) {
    $cache_hit = TRUE;
  }
  return $cache;
}


request_uri() returns the whole path including any GET arguments.