Tamped Gravel on fence posts and Frost Line?
A 24-inch depth works well for a 4-foot tall fence, but a fence that is 6 feet high should have posts sunk 3 feet deep or more. So dependent on the fence height above ground you may need to go with the 4x4x10.

Crushed rock works well for supporting fence posts because it contains rock pieces of a few inches in size, fine pebbles as well as fine gravel dust, which can pack together more tightly than using rocks of all the same size. If you use gravel instead, you can mix coarse masonry sand with the gravel for better compaction.

As for frost heaving/line, the best way to address this is how you dig your hole. If it is beveled at top then over time&mdash;whether concrete or gravel&mdash;it will pop out. So you want to create a lip at the top and that can be done by making the base of the hole wider than the top of the hole.

What causes frost heaving is the inward pressure and is comparable to squeezing a watermelon seed. How can that be prevented?  Answer: Dig a bell shaped hole to encase the post as much as possible. Fill hole with gravel leaving a few inches at top and compact that with earth; that helps with enclosing it as well and just looks better (to me). 

NOTE: Do not add more than 6" layers of gravel at a time and compact thoroughly between adding layers.

Wrong method in pic 1:



Correct way in pic 2: