Scenarios: Abusing a black hole
Black holes are not "made of matter". They are better described as structures of gravity/warped spacetime. However, they do grow when absorbing things with positive mass-energy. 

Antimatter still has positive mass-energy, so the black hole cheerfully gobbles it up. 

Now, exotic matter with negative energy density might indeed decrease the size of the black hole, akin to some form of enforced evporation process. (paper1, paper2) This is highly speculative since there are various theoretical reasons why exotic matter does not exist, and many kinds of hypothetical exotic matter that might work differently.