title: How can I exclude certain folders from being indexed by search engines in ASP.net when SSL is forced?
body: <p>Good day,</p>
  
  <p>I've done some research looking for this answer, but haven't had much luck. Hoping someone can help..</p>
  
  <p>The situation is that a site I'm working on (built in ASP.net) which forces SSL on most of their pages has some folders (ie. <code>site.com/dontindex</code> )containing files that definitely shouldn't be indexed by search engines. Google has links to these files in its index (ie. <code>https://www.site.com/dontindex/file.pdf</code>). </p>
  
  <p>My issue is that I have created a robots.txt file to disallow those folders from indexing, but from what I've read, that isn't going to prevent those files being indexed - as some of them might be referenced through secure pages. I'm thinking that only the non-secure pages are disallowed in this way. <strong>Q1) Is that even correct?</strong> </p>
  
  <p>When I tested <code>http://www.site.com/dontindex/file.pdf</code> against the new robots file in Google Webmaster Tools, wit came back as "Blocked by line 5: Disallow: /dontindex/", but when I tried <code>https://www.site.com/dontindex/file.pdf</code> it came back as "Not in domain".</p>
  
  <p>From what I can gather, I should have a second robots.txt file somewhere for the secure files/folders. I've read that if the site were running php, I could do some sort of a rewrite rule to cover this, but what to do in my ASP.net situation?? <strong>Q2) If it applies to me to have a second robots file (given that it's an ASP.net site), where should I put this file?</strong></p>
  
  <p>Thank-you for any help!</p>