codecov-io: ## [Current coverage](https://codecov.io/gh/scrapy/scrapy/pull/2393?src=pr) is 83.36% (diff: 100%)
> Merging [#2393](https://codecov.io/gh/scrapy/scrapy/pull/2393?src=pr) into [master](https://codecov.io/gh/scrapy/scrapy/branch/master?src=pr) will decrease coverage by **0.01%**

```diff
@@             master      #2393   diff @@
==========================================
  Files           161        161          
  Lines          8730       8730          
  Methods           0          0          
  Messages          0          0          
  Branches       1285       1285          
==========================================
- Hits           7279       7278     -1   
  Misses         1201       1201          
- Partials        250        251     +1   
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [de89b1b...626de8b](https://codecov.io/gh/scrapy/scrapy/compare/de89b1b562196fb8eec400c5b0d0ba9817a0271d...626de8b91c7a5585bf0becaf4841eeae7e38607a?src=pr)
kmike: It looks similar to https://github.com/scrapy/scrapy/pull/2001.

This is quite inconsistent in Scrapy: e.g. cache or ftp handler still won't use body to get response class.

I guess a part of the problem is that ResponseTypes.from_body is not based on any specification, so there was no strong desire to use body consistently - not using body was not seen as a bug because from_body may look like a hack. 

It seems the most up-to-date document is https://mimesniff.spec.whatwg.org, especially https://mimesniff.spec.whatwg.org/#identifying-a-resource-with-an-unknown-mime-type.

While this change looks fine, the fix is not complete, and it looks like a part of a larger problem.
redapple: @kmike , what do you mean by the fix not being complete?
Is it because `responsetypes.from_args()` does not follow whatwg?
Or because other uses of `from_args()` do not use the body either?

The aim here was to fix an issue at http decompression where, something else than the default `Response` type if there are hints in the body.
kmike: @redapple the patch looks good because it improves response handling in decompression middleware, so I'm fine with merging it after a rebase. 

The logic middleware uses to detect response type is still different from what browsers do, and Scrapy is inconsistent in mime sniffing it performs. I should have opened another ticket for that, but I found it while reviewing this PR, so I wrote it in a comment :) 
redapple: Maybe #2145 is so rare we don't need to care. (I have never seen it myself.)
diff --git a/scrapy/downloadermiddlewares/httpcompression.py b/scrapy/downloadermiddlewares/httpcompression.py
index 19d6345e4..eb00d8923 100644
--- a/scrapy/downloadermiddlewares/httpcompression.py
+++ b/scrapy/downloadermiddlewares/httpcompression.py
@@ -38,7 +38,7 @@ class HttpCompressionMiddleware(object):
                 encoding = content_encoding.pop()
                 decoded_body = self._decode(response.body, encoding.lower())
                 respcls = responsetypes.from_args(headers=response.headers, \
-                    url=response.url)
+                    url=response.url, body=decoded_body)
                 kwargs = dict(cls=respcls, body=decoded_body)
                 if issubclass(respcls, TextResponse):
                     # force recalculating the encoding until we make sure the

Disabling RedirectMiddleware results in HttpCompressionMiddleware errors
I wanted not to redirect `303` responses, but instead retry them.
From the docs, I thought I could achieve it through two settings:

```
REDIRECT_ENABLED = False
RETRY_HTTP_CODES = [301, 302, 307, 308, 500, 502, 503, 504, 408]
```

It ended up giving me errors on `HttpCompressionMiddleware`:

```
Traceback (most recent call last):
  File "twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "scrapy/core/downloader/middleware.py", line 53, in process_response
    spider=spider)
  File "scrapy/downloadermiddlewares/httpcompression.py", line 38, in process_response
    response = response.replace(**kwargs)
  File "scrapy/http/response/text.py", line 50, in replace
    return Response.replace(self, *args, **kwargs)
  File "scrapy/http/response/__init__.py", line 77, in replace
    return cls(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'encoding'
```
diff --git a/tests/test_downloadermiddleware_httpcompression.py b/tests/test_downloadermiddleware_httpcompression.py
index 7924fb3b5..5403e8f52 100644
--- a/tests/test_downloadermiddleware_httpcompression.py
+++ b/tests/test_downloadermiddleware_httpcompression.py
@@ -7,6 +7,7 @@ from scrapy.spiders import Spider
 from scrapy.http import Response, Request, HtmlResponse
 from scrapy.downloadermiddlewares.httpcompression import HttpCompressionMiddleware, \
     ACCEPTED_ENCODINGS
+from scrapy.responsetypes import responsetypes
 from tests import tests_datadir
 from w3lib.encoding import resolve_encoding
 
@@ -152,6 +153,20 @@ class HttpCompressionTest(TestCase):
         self.assertEqual(newresponse.body, plainbody)
         self.assertEqual(newresponse.encoding, resolve_encoding('gb2312'))
 
+    def test_process_response_no_content_type_header(self):
+        headers = {
+            'Content-Encoding': 'identity',
+        }
+        plainbody = b"""<html><head><title>Some page</title><meta http-equiv="Content-Type" content="text/html; charset=gb2312">"""
+        respcls = responsetypes.from_args(url="http://www.example.com/index", headers=headers, body=plainbody)
+        response = respcls("http://www.example.com/index", headers=headers, body=plainbody)
+        request = Request("http://www.example.com/index")
+
+        newresponse = self.mw.process_response(request, response, self.spider)
+        assert isinstance(newresponse, respcls)
+        self.assertEqual(newresponse.body, plainbody)
+        self.assertEqual(newresponse.encoding, resolve_encoding('gb2312'))
+
     def test_process_response_gzipped_contenttype(self):
         response = self._getresponse('gzip')
         response.headers['Content-Type'] = 'application/gzip'

pip install -e .
parse_log_pytest
requirements.txt
pytest pytest-cov pytest-xdist pytest-mock pytest-asyncio
apt-get update apt-get install -y gcc
3.9
requirements.txt
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
attrs==25.3.0
Automat==24.8.1
cffi==1.17.1
constantly==23.10.4
coverage==7.8.0
cryptography==44.0.2
cssselect==1.3.0
exceptiongroup==1.2.2
execnet==2.1.1
hyperlink==21.0.0
idna==3.10
incremental==24.7.2
iniconfig==2.1.0
jmespath==1.0.1
lxml==5.3.1
packaging==24.2
parsel==1.10.0
pluggy==1.5.0
pyasn1==0.6.1
pyasn1_modules==0.4.2
pycparser==2.22
PyDispatcher==2.0.7
pyOpenSSL==25.0.0
pytest==8.3.5
pytest-asyncio==0.26.0
pytest-cov==6.0.0
pytest-mock==3.14.0
pytest-xdist==3.6.1
queuelib==1.7.0
-e git+https://github.com/scrapy/scrapy.git@451f1474689a18d6a54630915c42172626624ef7#egg=Scrapy
service-identity==24.2.0
six==1.17.0
tomli==2.2.1
Twisted==24.11.0
typing_extensions==4.13.0
w3lib==2.3.1
zope.interface==7.2

name: scrapy
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - pip=25.0=py39h06a4308_0
  - python=3.9.21=he870216_1
  - readline=8.2=h5eee18b_0
  - setuptools=75.8.0=py39h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - tzdata=2025a=h04d1e81_0
  - wheel=0.45.1=py39h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - attrs==25.3.0
      - automat==24.8.1
      - cffi==1.17.1
      - constantly==23.10.4
      - coverage==7.8.0
      - cryptography==44.0.2
      - cssselect==1.3.0
      - exceptiongroup==1.2.2
      - execnet==2.1.1
      - hyperlink==21.0.0
      - idna==3.10
      - incremental==24.7.2
      - iniconfig==2.1.0
      - jmespath==1.0.1
      - lxml==5.3.1
      - packaging==24.2
      - parsel==1.10.0
      - pluggy==1.5.0
      - pyasn1==0.6.1
      - pyasn1-modules==0.4.2
      - pycparser==2.22
      - pydispatcher==2.0.7
      - pyopenssl==25.0.0
      - pytest==8.3.5
      - pytest-asyncio==0.26.0
      - pytest-cov==6.0.0
      - pytest-mock==3.14.0
      - pytest-xdist==3.6.1
      - queuelib==1.7.0
      - service-identity==24.2.0
      - six==1.17.0
      - tomli==2.2.1
      - twisted==24.11.0
      - typing-extensions==4.13.0
      - w3lib==2.3.1
      - zope-interface==7.2
prefix: /opt/conda/envs/scrapy

tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_no_content_type_header
tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_multipleencodings tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_request tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_encoding_inside_body tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_force_recalculate_encoding tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_gzip tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_gzip_app_octetstream_contenttype tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_gzip_binary_octetstream_contenttype tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_gzipped_contenttype tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_head_request_no_decode_required tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_plain tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_rawdeflate tests/test_downloadermiddleware_httpcompression.py::HttpCompressionTest::test_process_response_zlibdelate
scrapy/downloadermiddlewares/httpcompression.py
scrapy/downloadermiddlewares/httpcompression.py