Why isn't the mute key working for an external output device?
My guess is that the driver for your sound device may or may not implement all the expected sound controls on the keyboard. That is to say that they are implemented correctly by the system drivers (speakers, headphone socket) but these are not in the core sound processes - hence you need your driver to support them.

I have a Fiio USB headphone amplifier that I run from my MBP, and neither the mute nor the volume controls work on the keyboard. I suspect volume is applied by analogue circuits, not digital ones - so this would make sense. But I'm a touch surprised my mute doesn't work; maybe that is expected to be implemented by the same thing that controls the volume. (Edit: my device didn't come with a driver, but was recognised as a USB sound device automatically).

Long story short - I would guess that if there is a specific driver for your device, that installing that - or finding the control panel - might help. If you don't have a driver file, try googling for one (and obviously, only download from sources you trust).