Is there any way of removing (or unflagging) all of a user's flags?
You have to find out the event for expiring the roles, there you can unset the flag programmatically. However with other approach you can also add a check the role of user you are sending email to.

Here goes the steps:


Install rules_conditional module. This module allows to add
extra conditions inside the rules action.
Please check the screen shots. 


Configurations as in following screenshot.


Remember to have the nesting structure as in screenshot.