Why is the range of transponder digits 0 to 7 rather than 0 to 9?
The four-digit transponder code that is entered by a pilot is an octal number rather than a decimal number, and in the octal numbering system, only the digits 0-7 are valid.

As far as why, internally it's actually a 12-bit binary number and octal works really well since can be used as a "shortcut" for entering groups of three binary digits at a time (000 - 111, which is 0-7 in octal).

If it were designed today, with the additional computing power and software complexity available, it would be possible to use a decimal number even if it is stored as binary internally, just so that it is more familiar.  At the time that the system was created though, changing the dials directly set the code in hardware and doing the conversion would have been much more complex / difficult / expensive.

In short, it was convenient for the engineers and manufacturers, and just an oddity for the end users, so they left it that way.