Listing databases gives 'ERROR: invalid byte sequence for encoding “UTF8”: 0xe5 0xc6 0xf5'
Quick fix: change client encoding to match content
I got this working by changing the client encoding to match the server encoding. Changing the client encoding did the trick, as it prevented Postgres from encoding utf8 text when the encoding was cp1252. (I have no idea what the Collate stuff is.)
mydb=&gt; \encoding
WIN1252

mydb=&gt; \l
ERROR:  invalid byte sequence for encoding &quot;UTF8&quot;: 0xe5 0x6c 0x5f

mydb=&gt; \encoding utf8;

mydb=&gt; \l
                                                      List of databases
   Name    |    Owner    | Encoding |           Collate           |            Ctype            |      Access privileges
-----------+-------------+----------+-----------------------------+-----------------------------+-----------------------------
 mydb      |     myadmin | UTF8     | Norwegian Bokml_Norway.1252 | Norwegian Bokml_Norway.1252 | =Tc/myadmin            +
           |             |          |                             |                             | myadmin=CTc/myadmin+
           |             |          |                             |                             | my=c/myadmin
 postgres  | postgres    | UTF8     | Norwegian Bokml_Norway.1252 | Norwegian Bokml_Norway.1252 |
 template0 | postgres    | UTF8     | Norwegian Bokml_Norway.1252 | Norwegian Bokml_Norway.1252 | =c/postgres                +
           |             |          |                             |                             | postgres=CTc/postgres
 template1 | postgres    | UTF8     | Norwegian Bokml_Norway.1252 | Norwegian Bokml_Norway.1252 | =c/postgres                +
           |             |          |                             |                             | postgres=CTc/postgres

Why (an excursion into raw byte territory)
This took a while to dig into, but I used the tip from the comments section that \l is actually equivalent to a SQL query on the pg_catalog.pg_database. By switching which columns to list I was able to find the column that was proving problematic and by showing it as raw bytes I could pinpoint exactly where:

So it was the datcollate column that was proving problematic. According to the docs, this column is of type name and is LC_COLLATE for this database. Obviously there is something that escapes the eye, since there is nothing in the UTF8 encoded output that is &quot;non-ascii&quot;, but the output is a bit weird, since the locale name has some chars which seems to have disappeared: the &quot;å&quot; character in &quot;Norwegian Bokml_Norway.1252&quot; is missing (should be &quot;...Bokmål...&quot;). Is it just a coincidence?
Let's compare outputs of standard ascii with the output above:
$ echo -n Bokml | hexdump -C
00000000  42 6f 6b 6d 6c                                    |Bokml|
00000005

If you compare this with the raw output above you see that the psql output has an additional byte: e5 between 6d and 6c. This is not showing.
So this is how å is encoded in my UTF8 terminal:
$ echo -n Bokmål | hexdump -C
00000000  42 6f 6b 6d c3 a5 6c                              |Bokm..l|
00000007

$ printf &quot;%b&quot; &quot;\xc3\xa5&quot;
å

So å uses two bytes (c3a5) in UTF8, while above it is just one byte. Can it be å in WIN1252?
$ printf &quot;m%b&quot; &quot;\xe5&quot; 
m�

$ printf &quot;%b&quot; &quot;\xe5&quot; |  iconv -f cp1252 -t utf8 
å

Indeed, it was! So the problem is that the locale name is encoded in the internal postgres database as WIN1252, which seems to not be handled correctly when the server encoding is set to UTF8, as the client seems to assume that the data it receives will be all valid utf8.
Bonus: VIEW query issue fixed
Coincidentally, this also fixed another, related issue I was fixing, which happened when trying to list contents from a VIEW that contained UTF8 Norwegian characters:
mydb=&gt; select * from conversation_message limit 20;
ERROR:  character with byte sequence 0xf0 0x9f 0x91 0x8d in encoding &quot;UTF8&quot; has no equivalent in encoding &quot;WIN1252&quot;