How do I use rules to iterate over the subscribed users of a organic group to send email notifications?
There is a Flag action called Fetch users who have flagged [entity]. It will generate a list of users that you can then loop through and perform action on (e.g. send an email to). And since OG groups are entities, it should work (although I have not tested this). Also, beware that this might not scale if you have upwards of 1000 subscribers.

Reference: Nodeone's Learn the Rules framework. Checkout part 22 Flag and Rules: overview at the 4:00 minute mark.