Is water from the bathroom sink any dirtier than water from the kitchen sink?
All water coming into your house comes from the same single source, so unless you have two separate storage tanks for your kitchen taps and bathroom taps (highly unlikely) then the water will be identical.

The water should be potable without any additional filtering, but obviously you may want extra filtering if your water is particularly hard - i.e. has a high mineral content.

Where you site the filter raises another issue. If you site it just after the water enters your property and after the main stop cock then all the supply is filtered - including that used to flush toilets (which you could see as waste), but it does mean that the water going to boilers for heating and hot water will be filtered and reduce the build up of deposits in the heater itself (which is a good thing).