How to make HDRI map more subtle?
Set your HDRi images to light your scene as you'd normally would.
Enable Environment in the light passes section of the Layer Properties.

Enable Film Transparent, so that the background will render as an alpha channel and not as the world backgrund:

In the compositor use the output from the environment and blur it and darken it at will, then combine that result with your scene using an alpha over node.
That way the illumination for the scene will remain intact and the Background can be affected independently:

HDRi by GiantCowFilms
UPDATE: read this link to prevent a darker edge around the objects
(Images for version 2.7x)