What's the right size of gravel for fence posts?
It does not really matter or make much difference, so whichever is less expensive or more convenient.
The main thing (regardless of size) is to compact it well. Put in 50mm, tamp (or ram) with a flat-ended rod or stick. Put in another 50mm and repeat. Most people get tempted to put in much more at a time, and the compaction suffers as a direct result, making the post less stable.