Refusing the postdoc offer (email acceptance of the offer only)
Officially, you're always allowed to walk out. Particularly before signing any papers. Just make sure you let everyone know as soon as possible to avoid creating expectations and preparations.

Now, you're probably asking whether your peers will be happy if you walk out. Usually they will not appreciate anyone dropping a postdoctoral offer. If you understand exactly why they're pushy about you taking the offer, you will be in a better position to take this decision. 

Don't worry about making others happy. Do what you must.

Good luck!