title: How do I confirm the correct Robots.txt File is being used?
body: <p>I have various stag and dev domains for a project I am working on and ultimately will have the actual production environment. I have created two <code>robots.txt</code> files:</p>
  
  <ol>
  <li><code>robots.txt</code></li>
  <li><code>robots-nocrawl.txt</code></li>
  </ol>
  
  <p>The first is of course the standard file that I want to run on production and allow google to crawl.  I don't want my dev and stag domains indexed, so I want <code>robots-nocrawl.txt</code> fired when those domains are accessed, this file simply has <code>Disallow /</code>.  </p>
  
  <p>I have added the needed code the <code>.htaccess</code> file, see below, and moved the changes to my dev environment.  I can see that the nocrawl file is there, but pulling up the URL, but how to a check/confirm that the <code>robots-nocrawl.txt</code> file is the one being used when I am in dev/stag? I could not find a way to test this.</p>
  
  <pre><code>RewriteCond %{HTTP_HOST} ^stag\.mollywade\.([a-z\.]+)$ [OR]
  RewriteCond %{HTTP_HOST} ^dev\.mollywade\.([a-z\.]+)$ [OR]
  RewriteRule ^/robots\.txt$  /robots_nocrawl.txt [L]
  </code></pre>
  
  <p>Any help and/or suggestions would be greatly appreciated.</p>