Indexing the term of unpublished entities
For anyone who might encounter the same issue:

It looks like that it's a common issue in D7 and unfortunately going to be in D8, but hopefully the fix will be in core for D9.

Right now we have to use TEI module as a workaround for D7 and D8.

If you are installing this module into an existing site, visit admin/config/system/taxonomy-entity-index to rebuild the Taxonomy Entity Index.

Also note that in Views there will be some new fields/filters to work with, for instance instead of Content: Has taxonomy terms in filters section of views, you should use Taxonomy Entity Index: Has taxonomy terms on Nodes and for exposing the taxonomy terms also you should use Taxonomy Entity Index: Has taxonomy terms on Nodes not the default Content: Has taxonomy terms.