Is voltage reduced when load increases?
Yes, the output voltage of a battery decreases (roughly) in proportion to the current drawn from it. This behavior is usually characterized as an internal resistance of the battery, although its actual physical origin may be related more to the chemical properties of the battery than to the actual resistance of any conductive part of the battery.
For the second part of your question, you'd need to consult the data sheet of the battery you are using. This has been discussed in a previous question: Internal resistance of batteries?