Why Bluetooth audio works at so high latency?
This is just an approximation to what is happening: -

Streaming bluetooth packet size is about 1 kByte. If you are sending MP3 music at 192 kbits per second that's about 24 kB per second so 1 kB represents about one-24th of a second or about 42 ms.

You have to receive the full packet in order to check for errors so latency is tens of ms.

See also this stack overflow question and answer relating to maximum packet size.

As to why there is a packet size limit, consider two opposing requirements: -


Small packet sizes have a payload data size that is inefficient compared to the actual packet length (including headers and checksums) BUT, a small packet stands a good chance of being received without error.
A large packet size has a much more efficient use of payload data but stands a bigger chance of failure over a radio link.


I think Goldilocks had the same problem with the three bears.