Duplicate taxonomy terms from view
I think you're using a Node view instead of a Term (Taxonomy) view. So you're pulling in nodes and getting the tags associated with them. Since multiple nodes can have multiple tags, you're getting duplicates. I think the "Distinct" doesn't work because it's checking the Nodes to make sure they're distinct, not each field you add to the view.

So instead, try creating a view and choosing the Term type, instead of Node. I put together a very simple view without even using Distinct and I have no duplicated values, even though each term is definitely used on more than one node.

I don't know how you would do the count next to each term using Views, but I know the Advanced Taxonomy Blocks module does this.