Isn't the function PURGE ALL intended to purge all the unused assets in a scene? I tried this over and over, but it appears to not be working correctly from my point of view.
Several textures that are not used anywhere are not being purged.
I tried to close and reopen Blender, but when I open the file, it still again loads unused textures.
This is also very clear with a texture that I used several days ago for the world sphere, then I didn't use it anymore: it's still sitting there for nothing. Why? Nothing is recalling it: no material, no world, no scene. Why is it still there?
I know this happens... it's not fun.
If you switch your outliner window from View Layer to Blender File, you can look at the stuff in your file and right click and delete anything unneeded.

As you've noted it's usually images that need removing.
Technically, once unused they should be purged from your file by virtue of not being saved in the first place because they don't have any users.  I'm not sure what is marking these as used, honestly.  My suspicion is opening an image in a UV editor or something.