Will the battery from an iPhone 4 fit an iPhone 5
This is not something I would even try to do. 

The batteries on iPhones are designed for the particular model they're officially compatible with. Even an iPhone 5 battery is not compatible with an iPhone 5s or iPhone 5c. Likewise, the battery used for the iPhone 4 16GB and 32GB models is not officially compatible with the iPhone 4 8GB model (launched later).

In your example, they are also different in physical size and if you've ever seen the inside of an iPhone you will know how all the parts are designed to fit in perfectly with each other - even a fraction of a millimetre can make all the difference.

The above should be enough reason not to, but in addition to this, the actual specs of the batteries are different.

In a nutshell, it will be easier and cheaper in the end to just buy a replacement battery rather than do what you're suggesting. You can buy replacement batteries on eBay or specialist sites such as iFixit US, iFixit Europe or Macfixit Australia.

Also, if your iPhone 4 is still in working order, you never know when you may need it again (or need to lend it to someone else).

[EDIT]

I just realised I forgot to mention that you may be eligible for a replacement battery as part of an Apple recall program. To check if you are, refer to this info on Apple's website.