Why is hexafluoridocobaltate(III) the only common high-spin cobalt(III) complex?
This is a question that the crystal field theory cannot answer because of its inherent shortcomings.

The crystal field theory assumes point-type charges approaching the central metal from the coordinate axes leading to the destabilisation of those $\mathrm d$ orbitals that do not feature a nodal plane in the coordinate axes: $\mathrm{d}_{x^2-y^2}$ and $\mathrm{d}_{z^2}$. The others have zero contribution in all $x,y$ and $z$ direction and are thus not affected by point-sized ligands approaching from there. As simple as this model is, it only gives a crude approximation of why field splitting occurs and which orbitals are likely to end up higher in energy. For everything that involves quantitative discussions, the full molecular orbital approach is needed.

Below you will find the fullest molecular orbital scheme of an $\ce{[ML6]}$ complex including &sigma; and &pi; interactions. A simplified version without the &pi; interactions can be found here; you should attempt to understand it before adding the &pi; interactions to the picture. In this question, however, we need to understand the &pi; effects and thus I cannot leave them out.


Scheme 1: Molecular orbital scheme of an $\ce{[ML6]}$ complex featuring both &#x3c3; and &#x3c0; interactions.

The system we are looking at — $\ce{[CoF6]^3-}$ contains the &#x3c0;-basic fluoride ions (meaning that their $\mathrm p$ orbitals that can interact with the metal in a &#x3c0; symmetric manner, are fully populated). The interaction of the $\mathrm{t_{2g}}$ fluorine group orbital with the metal’s $\mathrm{t_{2g}}$ orbitals will create a bonding-antibonding interaction. With fluoride most likely having lower-lying orbitals, this interaction should raise the energy levels of cobalt’s $\mathrm{t_{2g}}$ orbitals which thereby turn into $\mathrm{t_{2g}{}^*}$ orbitals.

In this case, fluorido is the only strongly &#x3c0;-basic ligand you are considering in your comparison. And chloride does not form a hexachlorido complex like fluoride would but a tetrahedral tetrachloridocobaltate(III) for steric reasons. Thus, we don’t know whether the fact that fluoride is &#x3c0;-basic enough to raise the $\mathrm{t_{2g}}$ levels high enough so spin pairing does not occur is because of its &#x3c0; basicity alone or whether further elements come into play. Oxalato, ammin and aqua ligands have only weak &#x3c0;-basic effects while cyanido, if it displays &#x3c0; effects, is a &#x3c0; acid.