Restrict forum access based on user (user list)
If you use views or panels to display the forum, you can use their access controls which are quite flexible. Doesn't look like AF has built views for everything, so I think the easiest way would be to use Panels.
Create a page and under the page settings you have access rules that can be quite elaborate within the UI. You can use a naked layout and then just had the forum entities as content to display and render the full node.
Second Look:
Take a look at the Drupal Module Flexiaccess. It looks like exactly what is needed and has a nice UI to manage the users that have access to the individual forum with out writing/maintaining custom stuff:

The Flexi Access module will provide a simple and flexible interface
to the ACL (Access Control List) module. It will let you set up and
mange ACLs naming individual users that are allowed access to a
particular node.
The module let you manage access to individual nodes. The
administrator may grant individual users the right to: View (read),
update (edit/write), and delete the node.
The main idea behind Flexi Access is to allow flexible per user access
control for nodes without the complexity (or features) of Organic
Lists, and without having to create a lot of (overlapping) roles.

I see you using Flag to set up the &quot;subscription request&quot; then a moderator looks at the flagged users with the Flag and if approves add em to the ACL with flexiaccess?