Function to get the most frequently appearing pixel
There are numerous problems with the code. First of all, fread never returns EOF (which is -1). The loop should end when it reads less than 3 bytes.

Second, max_count is index, not a color value. I presume you meant rgb[max_count]. 

Test a file pointer returned by fopen for not being NULL.

Test a return value of fseek.

Avoid magic numbers. I suppose 54 is a size of certain header. Create a constant with a meaningful name. Better yet, define a header structure and take its sizeof.