Say I received a paper for review. Afterwards I discover that there are cited references that are not related to the discussed subject, or there are elements in the bibliography that are not referenced in the manuscript.

As a referee, is it ok to ask for the removal of such references? 

I ask because I have never heard of someone suggesting the removal of references, but on the contrary, I have many times heard of researchers self citing or citing friends just to enhance their numbers.