Entity Reference: User field not displaying unless user is referenced
Does that field display a link to the user profile? I think it does, and your users do not have the 'View user profiles' permission, so they cannot see the link to a page they don't have access to.