Access control in group based forums
After a small fight I managed to have it solved. There are few issues to have in mind like caching or preventing infinite recursion (you don't want to change the queries that you use inside the query altering function).

The code is based on TAC module.

function mymodule_query_term_access_alter($query) {
    static $mymodule_recursion_flag = FALSE;

    //prevent infinite recursion altering db_queries inside of this function
    if($mymodule_recursion_flag) return;

    // Take no action if there is no term table in the query.
    $alias = '';
    $tables =&amp; $query-&gt;getTables();
    foreach ($tables as $i =&gt; $table) {
        if (strpos($table['table'], 'taxonomy_term_') === 0) {
            $alias = $table['alias'];
        }
    }
    if (empty($alias)) {
        return;
    }

    //store once obtained results
    static $mymodule_tids = null;
    if($mymodule_tids == null){
        //set the recursion flag
        $mymodule_recursion_flag = TRUE;
        // Fetch a list of all terms the user should not see
        $mymodule_tids = mymodule_get_taxonomy_black_list();
        //unset the recursion flag
        $mymodule_recursion_flag = FALSE;
    }
    // No restrictions - user can list all terms
    if (empty($mymodule_tids)) {
        return;
    }
    // Otherwise, filter the terms
    else {
        $query-&gt;condition($alias . ".tid", $mymodule_tids, "NOT IN");
    }
}