is it possible to assign a filter to apply only on a specified material
Yes it is. Use material IDs to mask certain areas of the image.
I think the manual page gives you all the information and examples you need:

https://www.blender.org/manual/compositing/types/converter/id_mask.html