Difference: Caching and Baking
Caching is done with any type of physics (except for fluid simulation) while playing animation (files are being written to disk while animation is played) and everything will be cleared once any changes of settings are made. Note that you'll be able to edit settings of physics simulations on the fly, and previous cache will be cleared once animation playback started again (new values will be initialized).  It will not start if animation playback was started after starting frame of simulation.
Because of amount of computations done in real time, caching will last longer and speed of animation might be quite slow.


  Caching
  
  
  As animation is played, each physics system writes each frame to disk, between the simulation start and end frames. These files are stored in folders with prefix blendcache, next to the blend-file. Note that for the cache to fill up, one has to start playback before or on the frame that the simulation starts.  
  The cache is cleared automatically on changes - but not on all changes, so it may be necessary to free it manually e.g. if you change a force field.
  Cache and Bake  
  


Baking will pre-calculate animation taking into account every option set up and animation will not be changed unless bake freed or deleted. Once simulation is baked it is impossible to change most settings of objects used; meshes changes won't be taken into account and old results will be played instead.  

However, no computations will be done in real time and animation playback will be faster.


  Baking
  
  
  The system is protected against changes after baking.  
  If the mesh is changed the simulation is not calculated anew.
  Cache and Bake
  


Note also that baking physics simulations can be different in options depending on type of simulation, so several settings will depend on what you bake.