Why is kerosene preferred over other non-polar solvents to store sodium in?
Kerosene is a fraction from the distillation of petroleum, the one with boiling point 180° to 230°, containing hydrocarbons from $\ce{C_11}$ to $\ce{C_12}$. In reaction to what has been said in the commentaries (below the question), I note that mineral oil is a cloudy expression whereas kerosene seems to be the precise name of the more or less broader base material of this cloud. 

The best means to store alkali metals, which are all strong reductants, is an alkane because they are inert (as indicated by the name “paraffin”) and cannot be reduced.$^1$ The reason why one does not take the lower boiling fraction (gasoline 40° to 180°, $\ce{C_6}$ to $\ce{C_10}$), cyclohexane (b.p. 81°) or hexane (b.p. 69°) is obvious: it would be too evaporable, which is dangerous since its evaporation when someone forgets to close the bottle sufficiently, would lead to spontaneous ignition of the alkali metal (especially in the case of potassium) by contact with moist air. (Additionally, cyclohexane would anyway be unnecessary expensive.) The reason why the higher boiling fraction (light gas oil, 230 to 305°, $\ce{C_13}$ to $\ce{C_17}$) is not used is that this is much more greasy than kerosene.

It is strange that in the question benzene and carbon disulfide are proposed since they are both more expensive, very toxic and their b.p. is very low. — In the case of carbon disulfide this would be a paradigm of a very dangerous choice for three reasons: 1. concerning the boiling point (46°, extremely fiery), 2. concerning its extreme toxicity and 3. because it is not inert against reductants.



Footnote:

$^1$ In the case of lithium, which is much less reactive than the other alkali metals, the container should ideally be filled level with the top since it is swimming on the kerosene. But anyway the kerosene stays over as a protecting film on the lithium, which obstructs its reaction with moist air, even if it is not submerged.