Taxonomy terms view sorting results numerically
Try using Views Natural Sort

This allows you to emulate MYSQL's behavior of CAST() in order to treat the integers as pure values to sort by.

From the documentation:

Natural Sort also supports Numbers.

Normal Sort:

1 apple
10 apples
2 apples

Natural Sort:

1 apple
2 apples
10 apples