What exactly does each Mindstorms part do?
The best way to learn is to build the robots that come with instructions. By doing this you learn all kinds of ways to put the parts together to make functional models. 


  I bought a Lego Mindstorms (v2)


Assuming that by v2 you mean an NXT 2.0, if I remember correctly, there is an instruction manual with instructions for a simple wheeled/tracked vehicle, and several other robots are included in the software. Pick one (or all of them!) and start building! You'll notice that there really aren't that many different types of parts:  


Beams are the main structural component of the model. Pins and axles connect to them.
Pins connect beams together. They come in several shapes for different situations.
Axles rotate smoothly and support the wheels, motors, and gears. They can also be used as long pins. Use bushings or half-bushings to prevent axles from sliding out of their beam holes.
Connectors like this one allow you to attach beams and other parts at an angle to each other. These big pins are excellent for 90-degree angles.
Gears transfer/multiply torque. See this guide on gears.


For a simple wheeled/tracked robot, the basic design process is:


Build a sturdy chassis for the robot out of beams. Usually this frame is built around the motors. Since the motors are kinda weirdly shaped, consult other robots' instructions for tips on how to position them and attach them to beams.
Attach wheels to the motors using axles and bushings. You can change how fast the wheels move using programming or gears.
Attach the NXT to the top of the robot, so build up some structure from the frame to support it.
Wire the motors to the NXT outputs (A, B, and C), plug the thing into your computer, start programming.  


But all of this becomes more apparent and obvious after you've built a few robots from the instructions. Again, pay special attention to how beams are connected to each other to form a chassis, how the motors are mounted, and how different parts of the robot can be reinforced.