High-side switch for 24V controlled by logic level
While the comments are correct that you have exceeded the VGS(max) of the BSS84, there are indeed plenty of P-Chan Mosfets that will withstand more than 24V VGS(max).

This link to Digikey shows they are both available, and cheap.   

If you want to move to a newer device, then you should consider the ONSemi NUD3160 low side relay drivers which are fully self protecting (alas they don't make a high side driver). These save you components, board space and can be directly MCU port driven. 



OnSemi also have a smartFET high side driver. the NCV8460 which might be of use. I've not used these, but they include overvoltage, overcurrent and thermal shutdown along with open circuit load detection. Quite a capability for the price, though bigger than a SOT23.