title: How to disable robots.txt when you launch scrapy shell?
body: <p>I use Scrapy shell without problems with several websites, but I find problems when the robots (robots.txt) does not allow access to a site.
  How can I disable robots detection by Scrapy (ignored the existence)?
  Thank you in advance.
  <strong>I'm not talking about the project created by Scrapy, but Scrapy shell command: <code>scrapy shell 'www.example.com'</code></strong></p>