I am using Python request library to **scrape** robots.txt data from a list of URLs:

```
for url in urls:
    url = urllib.parse.urljoin(url, "robots.txt")
    try: 
        r = requests.get(url, headers=headers,  allow_redirects=False) 
        r.raise_for_status()
        extract_robots(r)
    except (exceptions.RequestException, exceptions.HTTPError, exceptions.Timeout) as err:
        handle_exeption(err)

```

In my list of urls, I have this webpage: <https://reward.ff.garena.com>. When I am requesting <https://reward.ff.garena.com/robots.txt>, I am directly redirected to <https://reward.ff.garena.com/en>. However, I specified in my request parameters that I don't want redirects `allow_redirects=False`.

How can I skip this kind of redirect and make sure I only have `domain/robots.txt` data calling my `extract_robots(data)` method?