Tethering blocked with message about contacting provider
The phone had separate APN settings for "MOBILE DATA" and "PERSONAL HOTSPOT".  Copying from the mobile data ones solved the problems.  

(With this provider, the date settings are often not be set automatically when switching SIM, at least on Android.  Also I don't remember the two settings to be separate.)