What exchange timescales are needed to cause decoupling in NMR?
You need to differentiate what gets averaged, a CS difference or a scalar coupling.

Methyl protons have a different chemical shift depending on the current conformation, but they rotate so fast this gets averaged out perfectly, and they have a very sharp signal. They show no coupling between themselves because it´s an $AB_2$ system with CS difference zero, so $A_3$ actually.

$J$ coupling to a methyl group (e.g. the ethyl group in ethanol) gives sharp lines and clear splitting, because the coupling varies very rapidly (with the methyl rotation) around a stable and finite value, and it only depends on the occupation of the spin states in both groups.

The coupling methylene to OH is inverted every time the proton gets exchanged for one with different spin state, so it averages to zero. The exchange happens so often that the methylene group still gives a sharp quartett from the methyl.

The OH group coalesces with the water peak. Because the exchange happens randomly and differently for each molecule, the coherence in your spin ensemble deteriorates, hence the line broadening. Note: It´s not a splitting that get´s averaged out, but a change in chemical shift.

If the proton exchange was even faster, the line would sharpen again at some point I guess. Just like the methyl protons themselves. If the exchange was even slower (milliseconds), the line would broaden so much it became invisible. If you have a really dry sample, the exchange becomes so slow it does not happen on the NMR time scale, the line becomes sharp, and you even see the $^3J$ coupling to the methylene group.