How to remove silicone from a porous wall?
this works fine 

http://www.rona.ca/en/paint---silicone-remover

you may have to find a supplier of it or something like it near you.  it takes a lot of time and patience on brick, and you will have to reapply it many times to get it all off, but if you are patient, it removes all traces of silicones.