What does the recommended values on a motor mean?
It really depends on the motor, but usually it is pretty self-describing:
Lets take a look at a spec sheet for a cheap motor: https://www.arduino.cc/documents/datasheets/T010160_DCmotor6_9V.pdf


Rated Voltage     6V DC  -- the rest of parameters in this spec sheet (RPM, current, etc..) apply when the motor is powered from 6V. You could power it from a different voltage but then the values in this spec sheet would not match.
Operating voltage 1.5-6.5V DC -- allowed voltage range; below 1.5V, the motor will not produce any torque; above 6.5V, the motor may overheat and likely fail
No load current ≤280mA -- if motor is not loaded, it will draw 0.28A (at 6V). So the motor wastes 6 * 0.28 = 1.7 watts on internal losses, friction, etc..
starting current ≤5A -- also known as stall current, this is the maximum current a motor will draw. This usually happens when motor cannot rotate, say because its shaft is blocked. Your motor driver must support this much current.


Other questions:


Does voltage correlate to speed or torque? -- In ideal motor, voltage linearly correlates with no-load speed. So half the voltage is half the speed. In the real motors this is not quite that simple because motor has internal resistance and friction, so if a voltage is too low motor will refuse to turn. Also, since motor when stalled acts like a resistor, hight voltage meaning higher stall current and higher stall torque.
Does amperage correlate to speed or torque? -- In ideal motor, amperage correlates with torque. In the real motors, the winding has non-zero resistance, so some amperage is wasted heating the motor up.
Is voltage how much voltage the battery must be rated? - the voltage refers to the voltage on the motor. This will be rated voltage for alkaline batteries, slightly more for lithium or lead-acid cells, slightly less if you use motor controller, etc... For the common case of alkaline cells connected directly to the motor via a mechanical switch, the battery rated voltage is a good approximation.
Is amperage how much amperage the motor will use ['pull' from the battery], or how much the battery must produce? Both. Motor will try to pull that much current. If the battery can produce it, great. If not, the motor will produce as much torque/speed as battery allows, and will stall if the battery simply does not have enough current. 


One notable exception is combination of high quality hobby RC vehicle motor and lithium-ion batteries -- if you stall that and have no protection, either motor or batteries will catch actual fire and possibly explode. So don't do that. (in the modern RC cars, the ESC - electronic speed controller -- is the most fragile part, and thus it will protect both motor and battery. A high quality ESC will limit the current / shutdown on overheat; a cheaper ESC will just fail permanently and then you will have to pay $150 to buy a new one :( )