title: how to restrict the site from being indexed
body: <p>I know this question was being asked many times but I want to be more specific.</p>
  
  <p>I have a development domain and moved the site there to a subfolder. Let's say from:</p>
  
  <pre><code>http://www.example.com/
  </code></pre>
  
  <p>To:</p>
  
  <pre><code>http://www.example.com/backup
  </code></pre>
  
  <p>So I want the subfolder to not be indexed by search engines at all. I've put robots.txt with the following contents in the subfolder (can I put it in a subfolder or it has to be at the root always, because I want the content at the root to be visible to search engines):</p>
  
  <pre><code>User-agent: *
  Disallow: /
  </code></pre>
  
  <p>Maybe I need to replace it and put in the root the following:</p>
  
  <pre><code>User-agent: *
  Disallow: /backup
  </code></pre>
  
  <p>The other thing is, I read somewhere that certain robots don't respect the robots.txt file so would just putting an .htaccess file in the /backup folder do the job?</p>
  
  <pre><code>Order deny,allow
  Deny from all
  </code></pre>
  
  <p>Any ideas?</p>