Using a LM311n in a class D amplifier as comparator
The comparator is possibly too slow (200 ns response time) and has an open collector output. The open collector output will undoubtedly contribute to the main problem of the exponential rise. This is also due to the 10k interacting with the o-scope input capacitance (maybe 10pF). Together they form a low pass filter of 3 dB point of about 1.6 MHz and therefore severely restrict the rise time.

I'd consider using a MAX999 comparator run on a 5V rail driving logic level MOSFETs feeding the speaker. To do this you'll need to arrange for your audio signal and triangle wave to be limited to within the 5V rail. I'm not ruling out a different comparator with a push-pull output but I don't think you'll ever get great success with driving a speaker (~ 8 ohms) directly.