Implementing softmax() in C The Softmax function, or normalized exponential function, is defined as:

$$
\sigma(z)_j = \frac{e^{zj}}{\sum^K_{k=1} e^{zk}} \text{for} \space j \space \text{in} \space [1, \dots, K]
$$

I am trying to implement this in C, starting with an answer here.

#include &lt;stdio.h&gt;
#include &lt;math.h&gt;
#include &lt;assert.h&gt;

static void softmax(float *input, int input_len)
{
    assert (input != NULL);
    assert (input_len != 0);
    int i;
    float m;
    /* Find maximum value from input array */
    m = input[0];
    for (i = 1; i &lt; input_len; i++) {
        if (input[i] &gt; m) {
            m = input[i];
        }
    }

    float sum = 0;
    for (i = 0; i &lt; input_len; i++) {
        sum += expf(input[i]-m);
    }

    for (i = 0; i &lt; input_len; i++) {
        input[i] = expf(input[i] - m - log(sum));

    }    
}

int main() {
   float arr[7] = {1.0, 2.0, 3.0, 4.0, 1.0, 2.0, 3.0};
   softmax(arr, 7);
   printf("%.3f %.3f %.3f %.3f %.3f %.3f %.3f", arr[0], arr[1], arr[2], arr[3], arr[4], arr[5], arr[6]);
}


Output:  

0.024 0.064 0.175 0.475 0.024 0.064 0.175


Any suggestion and comment are welcome.
Prefer double to float for calculations

Unless you're extremely constrained for space, or you have a platform with software floating-point (or very poor FP hardware), there's no reason to use float.

Reduce the scope of index variable

Modern C allows us to declare i local to the loops:

for (int i = 0;  i &lt; input_len;  i++)


Hoist the log(sum) calculation

Your compiler's optimizer will probably spot that log(sum) does not change, but why not help it?

const double log_sum = log(sum);
for (int i = 0;  i &lt; input_len;  i++) {
    input[i] = expf(input[i] - m - log_sum);
}


In fact, since m is also constant, we should hoist that, too:

const double scale = m + log(sum);
for (int i = 0;  i &lt; input_len;  i++) {
    input[i] = expf(input[i] - scale);
}