title: Usage of 'Allow' in robots.txt
body: <p>Recently I saw a site's robots.txt as follows:</p>
  
  <pre><code>User-agent: *
  Allow: /login
  Allow: /register
  </code></pre>
  
  <p>I could find only <code>Allow</code> entries and no <code>Disallow</code> entries.</p>
  
  <p>From <a href="https://support.google.com/webmasters/answer/6062596?hl=en" rel="nofollow">this</a>, I could understand robots.txt is nearly a blacklist file to <code>Disallow</code> pages to be crawled. So, <code>Allow</code> is used only to allow a sub part of domain which is already blocked with <code>Disallow</code>. Similar to this: </p>
  
  <pre><code>Allow: /crawlthis
  Disallow: /
  </code></pre>
  
  <p>But, that robots.txt has no <code>Disallow</code> entries. So, does this robots.txt let Google crawl all the pages? Or, does it allow only the specified pages tagged with <code>Allow</code>?</p>