title: How to prevent a development staging website, hosted on Azure, from being indexed by search engines
body: <p>Specific to Web Apps hosted on <code>Microsoft Azure</code>, is there a way to prevent the <code>mydomain.azurewebsites.net</code> URL from being indexed by search engines? I'm planning to use a web app as a staging website, and don't want it to accidentally get indexed. </p>
  
  <p>I know I could add a <code>robots.txt</code> file to the project with everything set to <code>no-index</code>, but I don't want to ever accidentally publish it to the production site (or alternatively, forget to publish it to the staging website). </p>
  
  <p>Is there a setting in Azure that will prevent the ".azurewebsites.net" domain from being indexed? Or if the <code>robots.txt</code> file is the only way, how do you keep it organized so that the right <code>robots.txt</code> file is published to staging and production, using <code>ASP.NET Core</code>.</p>