Can I downgrade from iOS5 to iOS4?
Depends. If your device has never been jailbroken then no, you cannot downgrade your device.

If your device has been jailbroken, then yes, maybe. Though it is definitely some work to edit your host file so that itunes checks a different server to sign your stuff and allow you to install a certain iOS version on your device. You got to have already given them (those jailbreak servers) your device information, which is typically strongly adviced by Cydia as well as any jailbreak instruction.