# SumSoftMaxWeight reduction (with LazyTensors)¶

Using the pykeops.numpy.Genred API, we show how to perform a computation specified through:

• Its inputs:

• $$x$$, an array of size $$M\times 3$$ made up of $$M$$ vectors in $$\mathbb R^3$$,

• $$y$$, an array of size $$N\times 3$$ made up of $$N$$ vectors in $$\mathbb R^3$$,

• $$b$$, an array of size $$N\times 2$$ made up of $$N$$ vectors in $$\mathbb R^2$$.

• Its output:

• $$c$$, an array of size $$M\times 2$$ made up of $$M$$ vectors in $$\mathbb R^2$$ such that

$c_i = \frac{\sum_j \exp(K(x_i,y_j))\,\cdot\,b_j }{\sum_j \exp(K(x_i,y_j))},$

with $$K(x_i,y_j) = \|x_i-y_j\|^2$$.

## Setup¶

Standard imports:

import time

import matplotlib.pyplot as plt
import numpy as np

from pykeops.numpy import LazyTensor as kf
from pykeops.numpy import Vi, Vj
from pykeops.numpy.utils import WarmUpGpu


Define our dataset:

M = 5000  # Number of "i" points
N = 4000  # Number of "j" points
D = 3  # Dimension of the ambient space
Dv = 2  # Dimension of the vectors

x = 2 * np.random.randn(M, D)
y = 2 * np.random.randn(N, D)
b = np.random.rand(N, Dv)

# KeOps implementation with the helper
WarmUpGpu()
start = time.time()
c = kf.sum((Vi(x) - Vj(y)) ** 2, axis=2)
c = kf.sumsoftmaxweight(c, Vj(b), axis=1)
print("Timing (KeOps implementation): ", round(time.time() - start, 5), "s")

# compare with direct implementation
start = time.time()
cc = np.sum((x[:, None, :] - y[None, :, :]) ** 2, axis=2)
cc -= np.max(cc, axis=1)[:, None]  # Subtract the max to prevent numeric overflows
cc = np.exp(cc) @ b / np.sum(np.exp(cc), axis=1)[:, None]
print("Timing (Numpy implementation): ", round(time.time() - start, 5), "s")

print("Relative error : ", (np.linalg.norm(c - cc) / np.linalg.norm(c)).item())

# Plot the results next to each other:
for i in range(Dv):
plt.subplot(Dv, 1, i + 1)
plt.plot(c[:40, i], '-', label='KeOps')
plt.plot(cc[:40, i], '--', label='NumPy')
plt.legend(loc='lower right')
plt.tight_layout();
plt.show() Out:

Warming up the Gpu (numpy bindings) !!!
Timing (KeOps implementation):  0.01656 s
Timing (Numpy implementation):  0.96769 s
Relative error :  7.354901217702609e-16
/home/bcharlier/keops/pykeops/examples/numpy/plot_test_softmax_numpy_helper.py:79: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
plt.show()


Total running time of the script: ( 0 minutes 11.977 seconds)

Gallery generated by Sphinx-Gallery
{"ft_lang_label":"__label__en","ft_lang_prob":0.5632383,"math_prob":0.9995365,"size":2414,"snap":"2019-51-2020-05","text_gpt3_token_len":776,"char_repetition_ratio":0.09170125,"word_repetition_ratio":0.06303725,"special_character_ratio":0.35584092,"punctuation_ratio":0.2238806,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":1.0000023,"pos_list":[0,1,2],"im_url_duplicate_count":[null,4,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-01-24T13:50:48Z\",\"WARC-Record-ID\":\"<urn:uuid:8c81a825-aad0-4061-b041-1526c31764b5>\",\"Content-Length\":\"23688\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:929b72e4-e315-4dd5-ae94-31355022d1ac>\",\"WARC-Concurrent-To\":\"<urn:uuid:66eb1e0d-d6a6-4731-8dca-61fdd383c9e5>\",\"WARC-IP-Address\":\"188.165.53.185\",\"WARC-Target-URI\":\"http://kernel-operations.io/keops/_auto_examples/numpy/plot_test_softmax_numpy_helper.html\",\"WARC-Payload-Digest\":\"sha1:M2POBBNTWJHF3MNNYWUB77XJDJVLYGS6\",\"WARC-Block-Digest\":\"sha1:NJBKOPX2NC4JI7DILQYW3HM5GVZBALZH\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-05/CC-MAIN-2020-05_segments_1579250620381.59_warc_CC-MAIN-20200124130719-20200124155719-00239.warc.gz\"}"}