SSRS Reports Manager Security Configuration
Normally in SSRS you would use the SSRS management interface (/ReportsManager IIRC) to set user roles/perms by folder. They are normally inherited by subfolders though you can override that. The role 'browser' is most appropriate for end users as it allows them to run / view reports but not much more. If you grant a user 'browser' access for folder  but not B then they will not be able to access folder B unless it is either a sub-folder of A (by default granted the same perms as A) or unless B is a sub-folder of another folder the user has 'Browser' access to.

You probably should not be granting domain users any rights directly with-in SQL Server. The various accounts that SQL Server itself uses as well as administrative accounts should be configured with-in SQL Server. End user access to SSRS is controlled by SSRS based on rules entered into SSRS.