Bottom bracket spacers
If you know you are doing this and accept the risk there is not really any reason not to do it. 

At the least you may get increased bearing wear. However, what I would be worried about is cracking the bottom bracket shell because you are moving the bearing and therefore the leverage point further outboard. That is going to increase the stress on the edge of the shell, even though the GXP cups are longer and have more threaded area than the Hollowtech units.