Is it safe to apply a small voltage to a GPIO pin in output mode?
No it is not safe. The MCU is going to malfunction at the long run. 

As an example, for AVR microcontrollers sink current must not be bigger than 20ma. So you have to make sure that the discharging current will not go higher.

I would recommend to connect the speaker through a driver which could be a transistor or an amplifier.