DOC: Disallow crawlers in development docs
(continiung the discussion in https://github.com/pandas-docs/pandas-docs-travis/pull/3#issuecomment-503148283)

Afaik we're currently not telling search engine which is the copy of the documentation they should be indexing. They seem to be smart enough, and they've been returning the right restults so far, but given that we're now changing the dev documentation to dev.pandas.io, it's probably worth to tell them not to index that copy.

It should be as easy as adding a `robots.txt` in the root of the dev docs repo with this content:
```
User-agent: *
Disallow: /
```
See: https://en.wikipedia.org/wiki/Robots_exclusion_standard

This can be easily done adding another line similar to the one used for the `CNAME`:
https://github.com/pandas-dev/pandas/blob/master/azure-pipelines.yml#L148

@TomAugspurger @jorisvandenbossche does this make sense?