Can one use a current transformer with turns ratio of 100:5A instead of 100:1A?
My question, when is it preferred to use CT of 100:1 turns ratio instead of a cheaper 100:5 amps?


When the meter has a full-scale of 1 A then you need a 100:1 CT so that you don't destroy the meter when the primary current goes over 20 A.
When the protection circuit has a full-scale of 1 A the 100:5 CT will cause it to trip at 20 A.
The VA rating of the CT must not be exceeded. If, for example, it's rated at 5 VA then the 5 A CT max burden is \$ R = \frac P {I^2} = \frac 5 {25} = 0.2 \ \Omega \$ whereas the 1 A CT has a max burden of \$ R = \frac 5 1 = 5 \ \Omega \$. This is probably the only thing that won't be a problem.


Is there a criteria for selecting the secondary rated current to be 5 instead of 1?

Yes, it has to match whatever it is driving - either metering or protection equipment.

From the comments:

For a relay CT, the ratio of the CT say it's 100:1. Does this mean that at 100 Amperes on the primary the relay would operate to trip the CB? or is the 100 Amps is the normal condition current and the tripping current is much higher?

The circuit breaker will have an actual trip setting of 5 A. If you use a 100:5 CT it will trip at 100 A primary current. If you put 3 primary turns through the  CT the CB will trip at 33.3 A primary current.
If you use a 100:1 transformer it will only put out 1 A and the 5 A trip on the CB will not be activated at 100 A. This will cause your protection circuit to fail.
Note: SI units named after a person have their symbols capitalised but are lowercase when spelled out. 'V' for volt, 'A' for ampere, 'K' for kelvin, 'Ω' (capital omega) for ohm, etc. Meanwhile 'k' is for kilo.
Capitals matter.