how to disable android headset/aux volume limit?
Sounds rather like something is overheating (hardware) especially the shorter times on repeated attempts. Presumably whatever that is is not in the speaker part of the circuit or when transmitting to bluetooth, but is specific to the headphone/aux jack part of the circuit.

So, probably a hardware bug (or "design flaw"), rather than a "limit" you can "disable."