Adjusting the voltage for a small speaker
From the P=V²/R I can calculate the voltage: V=sqrt(P*R)=3.46 volts


That's equivalent to 9.8 volts p-p and, if you are prepared to accept a distortion level of around 10% then that is what the MAX98357A amplifier can deliver into a 4 ohm load. If you are only prepared to accept 1% distortion then 2.5 watts is what the data sheet says you can expect.

However, if you are using it for music then be aware that a sine wave test doesn't really tell you much because the dynamic range (peak to RMS) for music is greater than 10 dB (compressed music) and usually much more for speech and other forms of music. For a sine wave the dynamic range is 3 dB (peak is 3 dB above RMS level).


  I get 2.5V. I assume this is how zero level looks like


Correct (for a 5 volt supply and MAX98357A bridge amplifier).