Should I set fence posts in dirt, gravel, crushed rock, or concrete?
Couple inches of gravel.  
Place post.  
Level, plumb.  
Fill with concrete.


Make sure that the concrete is graded away from the post so water runs off, this is an important step that lots of people overlook.  Taking the time now to make sure water moves away from the post, will save time and money later when the posts don't have to be replaced.

*Also make sure the concrete is above the ground around it.