Can I slim this UTF8-encoding program? Below is my entire program. You can read what it does thanks to the comments and specifications in particular.

My question is: can it be improved? Would it be possible, for example, to avoid writing a fwrite() inside each if? Is there a good pattern that can be implemented somewhere in this code?



The entire program is base on this UTF8 model and also studies the case in which a bit occurs in the 32nd position.




#include &lt;stdio.h&gt;
#include &lt;math.h&gt;
#include &lt;stdint.h&gt;

double log(double a);

/*
* This program reads 4 byte codepoints (in BIG ENDIAN) from a file strictly called "input.data" and creates another file called "ENCODED.data" with the relative encoding in UTF8.
* 
* In order to compile this file, in Unix, you need to add the -lm clause because the library math.h function log() requires it.
* For example: gcc encoding.c -o encoding -lm
*/
int main() {

    unsigned char bufferCP[4]; //Buffer used to store the codepoints
    unsigned char bufferOut[6]; //Buffer used to store the UTF8-encoded codepoints

    FILE *ptr, *out;
    ptr = fopen("input.data", "rb"); //r for read, b for bynary
    out = fopen("ENCODED.data", "wb");

    int elem = 0, bytesRead = 0;
    unsigned char mask = 0x3F; //Mask used to keep bits interesting for analysis
    uint32_t codepoint = 0; //A codepoint must be an unsigned 32 bit integer

    //--------------------File-Reading--------------------
    while ((elem = fgetc(ptr)) != EOF) {
        //Stores the character in the buffer
        bufferCP[bytesRead++] = (unsigned char) elem;

        if (bytesRead == 4) { //A codepoint is ready to be managed              

            //Builds a codepoint from the buffer. Reads it in BIG ENDIAN.
            for(int j=3; j&gt;=0; j--) {
                    codepoint &lt;&lt;= 8;
                    codepoint |= bufferCP[j];
            }
            //Searches the position of the most significant bit
            double logRes = (log(codepoint)/log(2)) + 1;
            int bitPos = (int) logRes;

            //--------------------UTF8-Encoding--------------------
            if (bitPos &lt;= 7) {
                bufferOut[0] = (unsigned char) codepoint; //No need to manage this codepoint
                fwrite(bufferOut, 1, 1, out);

            } else if (bitPos &lt;= 11) {
                bufferOut[0] = (codepoint &gt;&gt; 6) | 0xC0;
                bufferOut[1] = (codepoint &amp; mask) | 0x80;
                fwrite(bufferOut, 1, 2, out); 

            } else if (bitPos &lt;= 16) {
                bufferOut[0] = (codepoint &gt;&gt; 12) | 0xE0; 
                for(int i=1; i&lt;3; i++)
                    bufferOut[i] = ((codepoint &gt;&gt; 6*(2-i)) &amp; mask) | 0x80;
                fwrite(bufferOut, 1, 3, out);

            } else if (bitPos &lt;= 21) {
                bufferOut[0] = (codepoint &gt;&gt; 18) | 0xF0; 
                for(int i=1; i&lt;4; i++)
                    bufferOut[i] = ((codepoint &gt;&gt; 6*(3-i)) &amp; mask) | 0x80;
                fwrite(bufferOut, 1, 4, out);

            } else if (bitPos &lt;= 26) {
                bufferOut[0] = (codepoint &gt;&gt; 24) | 0xF8;
                for(int i=1; i&lt;5; i++)
                    bufferOut[i] = ((codepoint &gt;&gt; 6*(4-i)) &amp; mask) | 0x80;
                fwrite(bufferOut, 1, 5, out);

            } else if (bitPos &lt;= 32) {
                if (bitPos == 32)
                    bufferOut[0] = (codepoint &gt;&gt; 30) | 0xFE; //UTF8-encoding first byte would be: 11111111?
                else
                    bufferOut[0] = (codepoint &gt;&gt; 30) | 0xFC;

                for(int i=1; i&lt;6; i++)
                    bufferOut[i] = ((codepoint &gt;&gt; 6*(5-i)) &amp; mask) | 0x80;
                fwrite(bufferOut, 1, 6, out);
            }

            bytesRead = 0; //Variable reset
        }
    }

}
Efficient file I/O

By default, files opened with fopen() are buffered, meaning that not every call to fread() or fwrite() will result in a system call. Instead, the C library has an internal buffer and will try to read and write larger chunks at a time. However, you are still paying for the overhead of a regular function call each time you call fread() and fwrite(). To avoid this, it is best that you read and write in large chunks in your own code as well.

While you could try to read in the whole file at once, or even use technique like mmap() to memory map the file, you can already get very good performance by reading and writing blocks of say 64 kilobytes at a time. This avoids using a lot of memory.
Of course, you have to handle the last block not being exactly 64 kilobytes large, but that is quite easy to deal with.

Furthermore, fread() and fwrite() allow you to specify the size of an element and the number of elements you want to read, this comes in handy to ensure you read in a whole number of 4-byte codepoints.

I would structure your code like so:

uint32_t bufferIn[16384]; // 16384 4-byte code points = 64 kB
char bufferOut[65536];

size_t countIn;

while ((countIn = fread(bufferIn, sizeof *bufferIn, sizeof bufferIn / sizeof *bufferIn, ptr)) &gt; 0) {
    // There are countIn codepoints in the buffer
    for (size_t i = 0; i &lt; countIn; i++) {
         uint32_t codepoint = ...; // Convert bufferIn[i] to native endian here.

         // Write UTF-8 to bufferOut here.
         // If bufferOut is almost full, fwrite() it and start writing to it from the start.
    }
}

// Flush the remaining bytes in bufferOut here.


Don't use floating point math for integer problems

Avoid using floating point math when you are dealing with integers. It is hard to get it right, and converting int to double, doing some math operation, and then converting back again can be quite slow.

There are several ways to get the highest set bit in an integer. If you want a portable one, I recommend using one of the bit twiddling hacks. Sometimes compilers will even recognize such a bit twiddling hack and convert it to a single CPU instruction if possible.