title: Robots.txt Specific Exclusion
body: <p>Currently my <code>robots.txt</code> is the following</p>
  
  <pre><code>#Sitemaps
  Sitemap: http://www.baopals.com.com/sitemap.xml
  
  #Disallow select URLs
  User-agent: *
  Disallow: /admin/
  Disallow: /products/
  </code></pre>
  
  <p>My products have a lot of duplicate content as I pull over data from taobao.com and automatically translate it resulting in a lot of duplicate and low quality names which is why I just disallow the whole thing. However I manually change the titles on certain products and re-save them to the database and showcase them on the homepage with proper translations they just still get saved back to <code>/products/</code> and are lost forever when I remove them from the homepage.</p>
  
  <p>I'm wondering if it would be possible to allow the products that I save to the homepage with the updated translations still be indexed by google or am I forced to change the directory of the manually updated products?</p>