Why does this method return an unprintable character, how do I convert it to display correctly?
Because “h17” is an unprintable character, namely ASCII ETB. The 'h' in “h17” stands for “hexadecimal”, like the 0x usually used in C.

You should convert the value to an ASCII number if you want to print it, using for example Serial.print(payload.crc, 16); or – if you are not using Arduino core – (s)printf().



Edit: here is an implementation of my comment “sprintf() into a
buffer and dump the buffer to UDR0 one byte at a time”:

#include &lt;stdio.h&gt;

void print_byte_in_hex(uint8_t byte)
{
    // Print the byte in ASCII into an internal buffer.
    char buffer[8];
    sprintf(buffer, "%02x\r\n", byte);

    // Send it one byte at a time through the USART.
    for (char * p = buffer; *p; p++) {
        loop_until_bit_is_set(UCSR0A, UDRE0);
        UDR0 = *p;
    }
}