Machine Learning Golf: Multiplication
21 13 11 9 weights

This is based on the polarization identity of bilinear forms which in the one dimensional real case reduces to the polynomial identity:

$$ x\cdot y = \frac{(x+y)^2 - (x-y)^2}{4}$$

So y1 just computes [x+y, x-y] using a linear transformation, and y3 is just the absolute value of y1 as a preprocessing step for the next one: Then the "hard" part is computing the squares which I will explain below, and after that just computing a difference and scaling which is again a linear operation.

To compute the squares I use an exponential series \$s\$ which should be accurate for all integers \$\{0,1,2,\ldots,20\}\$ within around \$0.5\$. This series is of the form

$$ \text{approx_square}(x) = \sum_{i=0}^2 w_i \exp(0.0001 \cdot i \cdot x)$$

where I just optimized for the weights W2 (\$=(w_i)_i\$). This whole approximation comprises again just two linear transformations with an exponential activation sandwiched in between. This approach results in a maximal deviation of about 0.02.



function p = net(x)
% 9 weights
one = 1;&#32;
mone =-1;
zero = 0;
fourth = 0.25;
W1 = [1e-4, 2e-4];
W2  = [-199400468.100687;99700353.6313757];
b2 = 99700114.4299316;
leaky_relu = @(a,x)max(a*x,x);&#32;


% Linear
y0 = [one, one; one, mone] * x;

% Linear + ReLU
y1 = mone * y0;
y2 = [leaky_relu(zero, y0), leaky_relu(zero, y1)];

% Linear
y3 = y2 * [one; one];

% Linear + exp
y4 = exp(y3 * W1);&#32;

% Linear + Bias
y5 =  y4 * W2 + b2;

% Linear
y6 = [one, mone]*y5;
p = y6 * fourth;

end


Try it online!