View hdri in viewport "SOLID" Mode
In Blender 2.91.0 I see the so called Material Preview mode which allows what you're looking for.

There I have the option to use the built in HDRI's or installing my own one by conveniently using the little gear icon on that pop-up window.