codecov[bot]: # [Codecov](https://codecov.io/gh/scrapy/w3lib/pull/100?src=pr&el=h1) Report
> Merging [#100](https://codecov.io/gh/scrapy/w3lib/pull/100?src=pr&el=desc) into [master](https://codecov.io/gh/scrapy/w3lib/commit/0d365519ecac8da99fd051ea1a522486b13a7e8e?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.


```diff
@@           Coverage Diff           @@
##           master     #100   +/-   ##
=======================================
  Coverage   94.88%   94.88%           
=======================================
  Files           7        7           
  Lines         469      469           
  Branches       95       95           
=======================================
  Hits          445      445           
  Misses         16       16           
  Partials        8        8
```

| [Impacted Files](https://codecov.io/gh/scrapy/w3lib/pull/100?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [w3lib/http.py](https://codecov.io/gh/scrapy/w3lib/pull/100/diff?src=pr&el=tree#diff-dzNsaWIvaHR0cC5weQ==) | `100% <100%> (ø)` | :arrow_up: |

kmike: Could you please rebase it, to fix CI? Code looks good to me.
diff --git a/w3lib/http.py b/w3lib/http.py
index accfb5d..c7b94a2 100644
--- a/w3lib/http.py
+++ b/w3lib/http.py
@@ -78,7 +78,7 @@ def headers_dict_to_raw(headers_dict):
     return b'\r\n'.join(raw_lines)
 
 
-def basic_auth_header(username, password):
+def basic_auth_header(username, password, encoding='ISO-8859-1'):
     """
     Return an `Authorization` header field value for `HTTP Basic Access Authentication (RFC 2617)`_
 
@@ -95,5 +95,5 @@ def basic_auth_header(username, password):
         # XXX: RFC 2617 doesn't define encoding, but ISO-8859-1
         # seems to be the most widely used encoding here. See also:
         # http://greenbytes.de/tech/webdav/draft-ietf-httpauth-basicauth-enc-latest.html
-        auth = auth.encode('ISO-8859-1')
+        auth = auth.encode(encoding)
     return b'Basic ' + urlsafe_b64encode(auth)

allow to customize encoding in w3lib.http.basic_auth_header 
See https://github.com/scrapy/scrapy/pull/2530
diff --git a/tests/test_http.py b/tests/test_http.py
index 453624f..01f903e 100644
--- a/tests/test_http.py
+++ b/tests/test_http.py
@@ -1,3 +1,5 @@
+# -*- coding: utf-8 -*-
+
 import unittest
 from collections import OrderedDict
 from w3lib.http import (basic_auth_header,
@@ -14,6 +16,13 @@ class HttpTests(unittest.TestCase):
         self.assertEqual(b'Basic c29tZXVzZXI6QDx5dTk-Jm8_UQ==',
             basic_auth_header('someuser', '@<yu9>&o?Q'))
 
+    def test_basic_auth_header_encoding(self):
+        self.assertEqual(b'Basic c29tw6Z1c8Oocjpzw7htZXDDpHNz',
+                basic_auth_header(u'somæusèr', u'sømepäss', encoding='utf8'))
+        # default encoding (ISO-8859-1)
+        self.assertEqual(b'Basic c29t5nVz6HI6c_htZXDkc3M=',
+                basic_auth_header(u'somæusèr', u'sømepäss'))
+
     def test_headers_raw_dict_none(self):
         self.assertIsNone(headers_raw_to_dict(None))
         self.assertIsNone(headers_dict_to_raw(None))

pip install -e .
parse_log_pytest
pytest
pytest pytest-cov
apt-get update apt-get install -y gcc
3.9
pytest --no-header -rA --tb=line --color=no -p no:cacheprovider -W ignore::DeprecationWarning
coverage==7.8.0
exceptiongroup @ file:///croot/exceptiongroup_1706031385326/work
iniconfig @ file:///home/linux1/recipes/ci/iniconfig_1610983019677/work
packaging @ file:///croot/packaging_1734472117206/work
pluggy @ file:///croot/pluggy_1733169602837/work
pytest @ file:///croot/pytest_1738938843180/work
pytest-cov==6.0.0
six==1.17.0
tomli @ file:///opt/conda/conda-bld/tomli_1657175507142/work
-e git+https://github.com/scrapy/w3lib.git@ac1b7212b5f9badf2b35aaf6e094d839fb3ba858#egg=w3lib

name: w3lib
channels:
  - defaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2025.2.25=h06a4308_0
  - exceptiongroup=1.2.0=py39h06a4308_0
  - iniconfig=1.1.1=pyhd3eb1b0_0
  - ld_impl_linux-64=2.40=h12ee557_0
  - libffi=3.4.4=h6a678d5_1
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.4=h6a678d5_0
  - openssl=3.0.16=h5eee18b_0
  - packaging=24.2=py39h06a4308_0
  - pip=25.0=py39h06a4308_0
  - pluggy=1.5.0=py39h06a4308_0
  - pytest=8.3.4=py39h06a4308_0
  - python=3.9.21=he870216_1
  - readline=8.2=h5eee18b_0
  - setuptools=75.8.0=py39h06a4308_0
  - sqlite=3.45.3=h5eee18b_0
  - tk=8.6.14=h39e8969_0
  - tomli=2.0.1=py39h06a4308_0
  - tzdata=2025a=h04d1e81_0
  - wheel=0.45.1=py39h06a4308_0
  - xz=5.6.4=h5eee18b_1
  - zlib=1.2.13=h5eee18b_1
  - pip:
      - coverage==7.8.0
      - pytest-cov==6.0.0
      - six==1.17.0
prefix: /opt/conda/envs/w3lib

tests/test_http.py::HttpTests::test_basic_auth_header_encoding
tests/test_http.py::HttpTests::test_basic_auth_header tests/test_http.py::HttpTests::test_headers_dict_to_raw tests/test_http.py::HttpTests::test_headers_dict_to_raw_listtuple tests/test_http.py::HttpTests::test_headers_dict_to_raw_wrong_values tests/test_http.py::HttpTests::test_headers_raw_dict_none tests/test_http.py::HttpTests::test_headers_raw_to_dict
w3lib/http.py
w3lib/http.py