What decides the order of priority of functional groups in organic compounds?
It’s an arbitrary agreement by IUPAC, although note
that there is some correlation between the
oxidation state of the carbon and the priority
(more oxidized groups tend to be higher
priority). Generally, the functional group which shows its characteristics more dominantly even when present with other group gets a higher priority.