<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="request.meta['handle_httpstatus_all'] = True
dfd = self.crawler.engine.download(request, info.spider)"><pre class="notranslate"><code class="notranslate">request.meta['handle_httpstatus_all'] = True
dfd = self.crawler.engine.download(request, info.spider)
</code></pre></div>
<p dir="auto">When Media Pipeline schedules request it attached <code class="notranslate">handle_httpstatus_all</code> meta which prevents Redirect Middleware from doing it's job. This breaks download of images that require redirect, mostly urls that need to redirect from <code class="notranslate">http</code> to  <code class="notranslate">https</code>.</p>
<p dir="auto">Maybe implement a setting to enable redirects in this pipeline?</p>
<p dir="auto">Basically, what's happened is that my spider is unable to download the files because the file_urls provided are actually redirected to the final download link. However, because of the following code, the redirect download middleware is effectively disabled, which makes the download fail.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="def _check_media_to_download(self, result, request, info):
        if result is not None:
            return result
        if self.download_func:
            # this ugly code was left only to support tests. TODO: remove
            dfd = mustbe_deferred(self.download_func, request, info.spider)
            dfd.addCallbacks(
                callback=self.media_downloaded, callbackArgs=(request, info),
                errback=self.media_failed, errbackArgs=(request, info))
        else:
            request.meta['handle_httpstatus_all'] = True
            dfd = self.crawler.engine.download(request, info.spider)
            dfd.addCallbacks(
                callback=self.media_downloaded, callbackArgs=(request, info),
                errback=self.media_failed, errbackArgs=(request, info))
        return dfd"><pre class="notranslate"><code class="notranslate">def _check_media_to_download(self, result, request, info):
        if result is not None:
            return result
        if self.download_func:
            # this ugly code was left only to support tests. TODO: remove
            dfd = mustbe_deferred(self.download_func, request, info.spider)
            dfd.addCallbacks(
                callback=self.media_downloaded, callbackArgs=(request, info),
                errback=self.media_failed, errbackArgs=(request, info))
        else:
            request.meta['handle_httpstatus_all'] = True
            dfd = self.crawler.engine.download(request, info.spider)
            dfd.addCallbacks(
                callback=self.media_downloaded, callbackArgs=(request, info),
                errback=self.media_failed, errbackArgs=(request, info))
        return dfd
</code></pre></div>
<p dir="auto">And here is the check in the redirect middleware that disables it:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="if (request.meta.get('dont_redirect', False) or
                response.status in getattr(spider, 'handle_httpstatus_list', []) or
                response.status in request.meta.get('handle_httpstatus_list', []) or
                request.meta.get('handle_httpstatus_all', False)):
            return response
"><pre class="notranslate"><code class="notranslate">if (request.meta.get('dont_redirect', False) or
                response.status in getattr(spider, 'handle_httpstatus_list', []) or
                response.status in request.meta.get('handle_httpstatus_list', []) or
                request.meta.get('handle_httpstatus_all', False)):
            return response

</code></pre></div>
<p dir="auto">My question is: What is the point of enabling the handling of httpstatus_all, when it effectively disables all of the checks?</p>