Why are bytes treated like the base unit?
For historical reasons.

In the early days and even so today a byte represented the most commonly used data.

A byte represents one character in ASCII.

A byte was and still is the size of the instruction for the most commonly used opcodes at the assembly level. Having the most common instructions in one byte saved the most amount of memory.

It is a power of two so 8 is better than 7,6, and 5.

You can get two BCD digits in one byte.