Improve query: find users who are neither friends nor fans of a user
This query will give you the same results

SELECT username FROM users 
WHERE username NOT IN (SELECT you FROM friends WHERE friend = 'user1') 
AND username NOT IN (SELECT friend FROM friends WHERE you = 'user1')
AND username &lt;&gt; 'user1'


and the SQLFiddle

If you look at the execution plan, it looks like my query grabbed 1 less row throughout the execution and it didn't have to use a temporary or a join buffer.  

This Query is more readable than yours, assuming this is what you are trying to do.