Deadbeef : finding all words made of hexadecimal digits
It is customary to put helper functions first and main() last, to avoid having to write forward declarations like int process_file(FILE* file);.

process_file() is a very generic name.  I suggest renaming it to print_hex_words().

The process_file() function returns an error code.  Therefore, the responsibility for printing any error message for I/O errors should lie with main().

You assume that words are delimited by whitespace, and have neglected to deal with punctuation.

Your algorithm is very tedious.  Instead of using getc() to read a byte at a time, use fscanf() to read a whitespace-delimited word at a time.  To skip to the end of a sequence consisting solely of A-F characters, use strspn(…, "ABCDEFabcdef").

#define xstr(s) str(s)
#define str(s) #s

int print_hex_words(FILE* file) {
    char word_buf[MAX_LEN + 1];
    while (1 == fscanf(file, "%" xstr(MAX_LEN) "s", word_buf)) {
        char *word, *end, *trail_punct;

        /* Skip leading punctuation */
        for (word = word_buf; ispunct(*word); word++);

        end = word + strspn(word, "ABCDEFabcdef");

        /* Skip trailing punctuation */
        for (trail_punct = end; ispunct(*trail_punct); trail_punct++);

        if (word != end &amp;&amp; *trail_punct == '\0') {
            /* NUL-terminate the word and convert it to lowercase */
            *end = '\0';
            for (end = word; (*end = tolower(*end)); end++);

            printf("%s\n", word);
        }
    }
    return ferror(file);
}


Instead of … | sort | uniq, you can use … | sort -u.