I've been working my way through an undocumented TCP-based protocol by monitoring traffic with tcpdump, Wireshark, and some Python packages (scapy, numpy) with good luck so far, but I've hit a wall with what I think is some sort of checksum/hash/CRC.Watching a client binary, it sends its first hello sort of message, gets back a response from the server with some basic things (server version, etc.), then sends a set of commands in the subsequent messages.  It's that second and subsequent message type I can't figure out.  Here's an example:2f c6 00 f2 00 36 01 08  56 cc 80 c4 00 00 00 0000 00 00 00 01 04 72 6f  6f 74 00 68 6f 73 74 6e61 6d 65 00 6c 6d 67 72  64 00 2f 64 65 76 2f 7074 73 2f 30 00 00The first 22 bytes are what I'm calling the header, since it's a fixed-length segment and after that it's all just null-terminated ASCII strings.  The bytes of that header:1st: always 0x2f2nd: looks like a sort of sum over the message bytes but I'm not 100% on that part3rd and 4th: total mystery to me and the reason for this post5th: always 0x006th: total number of bytes in the message7th-8th: always 0x01089th-12: four-byte unix timestamp13th-20th: always 0x0021st-22nd: always 0x0104My question is about the 3rd and 4th byte: what method is the client using to come up with those values, and/or what should I try to figure it out myself?I can test a wide variety of values by varying the hostname, username, system time, and TTY the client is running under, so I can easily gather gobs of example packets if needed.  So far I've tried a statistical approach with that by looking at a histogram of what values those bytes take on across a wide range of inputs.  I haven't discovered too much from that, except:3rd byte spans 0 - 63 (i.e., its first two bits are always 0)4th byte spans 0 - 255neither byte ever takes on the values 10 - 13 (there's a gap in an otherwise evenly-spread histogram on these specific values)Another method was to give it a few specific values of hostname to see how it behaves.  These things DO change the value of both of those mystery bytes:Switching the order of message bytesInverting a bit in a particular position in two different bytesI then tried crunching through different types of checksums I've read about (variations on BSD and Fletcher) as well as CRCs (trying a 15-bit polynomial to produce a 14-bit remainder, but, is that actually a thing that's done?) with no luck so far.  Any ideas?
What checksum algorithm is this network protocol using?
linux;protocol;networking;wireshark
Got it!The algorithm is a CRC with the following parameters, stored in those two bytes as a big-endian short:Polynomial: 0x2e97Xor In Value: 0Xor Out Value: 0Reflect Input: TrueReflect Out: TrueTo figure it out, I used my packet capture data, CRC RevEng, and some shell scripting to glue it together.  I then used pycrc to calculate the check values on the fly to make my own network requests check out OK on the server. I'm considered my question answered since this method reliably gets the server to acknowledge my custom requests, but what Jason and ebux recommended for reverse engineering the binaries still looks to be the only way I'd be able to pin it down any further.  For example, there were a few different CRCs that fit all the data I'm working with. I learned quite a bit with using gdb and objdump, but quickly got in over my head.  Time will tell if I've just delayed the inevitable, I suppose.More details:First in Wireshark I selected a good range of packets, some with the same length, some with differing lengths.  For each I exported the raw binary into a file.  (In the Packet Details frame, right-click the Data display, click Export selected packet bytes....)  Since I didn't know for sure how the bytes were organized, but I had a good guess of which were the CRC, I made modified versions of the files with the check bytes appended instead of at the beginning, with some clunky dd usage show below.  The variant that finally worked was putting the bytes at the end in a little-endian format that reveng would understand.  In a bash shell:dd if=example bs=1 skip=4 > example_crc_reversedd if=example bs=1 count=1 skip=3 >> example_crc_reversedd if=example bs=1 count=1 skip=2 >> example_crc_reverseThese modified files were then fed into reveng.  Since I didn't even know for sure the width of the possible CRC polynomial, I looped from 1 to 16 to see what it'd find.  In bash again:for width in $(seq 1 16); do reveng -w $width -s -f example*_crc_reverse; doneEven with many packets it finds a number of non-standard CRCs that fit the data, but the one above is the highest-order one that doesn't require strange xor values.  The ones found in the reveng output:width=1  poly=0x1  init=0x0  refin=false  refout=false  xorout=0x0  check=0x1  name=(none)width=1  poly=0x1  init=0x1  refin=false  refout=false  xorout=0x1  check=0x1  name=(none)width=1  poly=0x1  init=0x0  refin=true  refout=true  xorout=0x0  check=0x1  name=(none)width=1  poly=0x1  init=0x1  refin=true  refout=true  xorout=0x1  check=0x1  name=(none)width=3  poly=0x5  init=0x0  refin=true  refout=true  xorout=0x0  check=0x0  name=(none)width=4  poly=0x7  init=0x0  refin=true  refout=true  xorout=0x0  check=0xb  name=(none)width=4  poly=0x7  init=0xd  refin=true  refout=true  xorout=0xb  check=0xb  name=(none)width=10  poly=0x381  init=0x000  refin=true  refout=true  xorout=0x000  check=0x032  name=(none)width=11  poly=0x083  init=0x000  refin=true  refout=true  xorout=0x000  check=0x032  name=(none)width=11  poly=0x083  init=0x781  refin=true  refout=true  xorout=0x40f  check=0x032  name=(none)width=13  poly=0x058d  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x1c1f  name=(none)width=14  poly=0x2e97  init=0x0000  refin=true  refout=true  xorout=0x0000  check=0x3076  name=(none)width=14  poly=0x2e97  init=0x258d  refin=true  refout=true  xorout=0x2c69  check=0x3076  name=(none)(I thought the 14-bit one with no xorin/xorout step made the most sense to use, but I don't have a good argument why.) In Python, with a binary request prepared as a list of integer byte values, pycrc can calculate the check value like this:crc = pycrc.Crc(width=14, poly=0x2e97, reflect_in=True, xor_in=0, reflect_out=True, xor_out=0)crc_val = crc.table_driven(binary)The calculated crc_val from pycrc is already stored with the endianness expected by the server, so it can be placed as-is a the beginning of the message.