Will I lose my game saves when I downgrade to iOS 4 from iOS 5?
iTunes will backup the data, but there is another option.  You can also bring your game saves back to your downgraded firmware, but it will take some work.

Here is a walkthrough focusing specifically on Angry Birds, but you could use the same process for other games as well.