Non-numeric Diffie-Hellman?
Diffie-Hellman relies on a mathematical problem on positive integers. To use it with bytes you just have to convert the bytes to - or use the bytes as - an integer. Usually this would be a unsigned big-endian (or network order) integer.

For Diffie-Hellman the parameters consist of the modulus and the base. The public value could be 1024 bits (128 bytes). 1024 bit cryptography is considered "legacy strength" for discrete logarithm related problems though.



With Java for instance the conversion can be as easy as new BigInteger(1, data) where 1 indicates a positive integer and data is a byte array.