Sql Query with In Operator optmization
Look at

SELECT DISTINCT SiteUserID  
FROM SiteUserBroker

EXCEPT 

SELECT t1.SiteUserID
FROM SiteUserBroker t1
LEFT JOIN SiteUserBroker t2 ON  t1.BrokerID = t2.BrokerID
                            AND t2.SiteUserID = @SiteUserID
WHERE t2.SiteUserID IS NULL