Can't add contextual Filter (taxonomy term ID (with depth)) to a View (User)
In D7, you can only do that with the node table (the content).

If you want to have a working solution for every entity of your system, you have to use this module.

Once installed, don't forget to rebuild the index of your entities, then, in views, you'll have the possibility to add a new relationship ('Taxonomy Entity Index: Taxonomy terms on User') and a filter with depth ('Taxonomy Entity Index: Has taxonomy terms on User (with depth and indexed in taxonomy_entity_index)').