Physical significance of group delay
As already pointed out in other answers, the group delay (at a certain frequency) is the delay of the envelope of a narrowband signal centered at that frequency. Note that the term group delay is only meaningful for a few special signals (see also this answer). In general, it's more useful to directly consider a system's phase response.

The relevance of phase in communication systems is that a non-linear phase on the channel causes intersymbol interference (ISI), i.e. the symbols get mixed up in time and influence each other, and cannot be detected properly anymore by the receiver. ISI is usually mitigated by using an adaptive equalizer prior to detection.