In this letter, we address the effect of coupling many molecules into a cavity on the coherence lifetimes of the polaritonic states. The coherences of a model light-matter Hamiltonian with many molecules were examined and exact quantum dynamics, based on the hierarchical equation of motion (HEOM) formalism, is performed on this model Hamiltonian. We demonstrate through numerical results from HEOM that the coherence lifetimes increase with the collective light-matter coupling strength. Moreover, we explain the enhancement in the polariton's coherence lifetime using Fermi's golden rule (FGR) argument in the frequency domain, and this accounts for the scaling of the coherence lifetimes with respect to the number of molecules and the single molecule light-matter coupling strength.