Is a 3-5 second delay normal for a Retina MacBook Pro to wake up from sleep?
After the rMBP or the MacBook Air are left sleeping for one hour, they enter a deep sleep mode in order to conserve battery.  Traditional Macs without SSDs do not enter this mode, so they might awake from sleep a bit faster.

To answer your question, that is normal behavior.