Why does volume above 80% break my speaker (PAM8403, 3w 4ohm, 18650)?
What you are describing is distortion.  Your speaker isn't &quot;breaking.&quot;  If it broke, it wouldn't work anymore afterwards.
There are a couple of possible causes for distortion in your case:

The battery can't supply the needed current.
Your audio signal is too high.

First the battery:

You are using an 18650 cell to power your amplifier.
You are using a voltage booster to get 5V for the amplifier.
You have  two 3 watt speakers.
The amplifier loses a little power as heat, maybe 10 or 15 percent.

You need about 7 watts at 5 volts.  That's about 1.2 amperes from the voltage booster.  That's closer to 1.6 amperes from the battery.  Without specifications, I can't tell you if that is too much for the 18650 you have.
Your voltage booster is rated for 2 amperes.  It ought to be OK.

Turn the volume down.
Measure the battery voltage and the 5V output.  Write them down.
Turn the volume up to just before the distortion starts.
Measure the battery voltage and the 5V output again.  Write the values down.

They should both be stable - neither value should drop much.

If the battery voltage drops but the 5V stays up, then the battery isn't up to the task.
If the battery voltage is stable but the 5V drops, then the booster isn't up to the task.
If both drop, then the battery is too weak and the regulator might be too weak.  Use a better battery, test again.

If the battery voltage and the 5V are both good then the problem is that your signal is too high.
The PAM8403 IC used in your modules amplify by a little over 100 times.
A signal input of 1V peak to peak would give an output voltage of 100 volts peak to peak.
That's impossible, of course.  The module only has a 5V power supply - the output can only be a maximum of 5V peak to peak.
You have to keep your input signal low enough that the output signal after the amplification will be less than 5V peak to peak. That would be about 50 millivolts peak to peak.
Typical earphone outputs such as your smartphone or MP3 player will have an output level of 200 millivolts peak to peak - or more.
You can reduce the amplification by changing two resistors.
Here's your PAM8403 module:

There are two resistors on there that are marked &quot;103&quot; (black with white lettering.) Those are 10 kiloohm resistors.
Replace them with 47 kiloohm resistors.  That's an &quot;eyeballed&quot; value that ought to lower the amplification to a reasonable level.  If you still get distortion at high volume, just use a couple of bigger resistors.
Rather than replace the resistors, you could just put a resistor in series with each input. That will also lower the amplification.  It will also mess with the bass sound, though.

The 10k resistors are smaller than the resistors recommended in the PAM8403 datasheet.  That gives it more amplification than it is supposed to have, and leaves it more susceptible to distortion and maybe other problems.