What is the point of multinomial vs argmax evaluation of accuracy?
What is the purpose of evaluating prediction accuracy using multinomial instead of the straight up argmax? 

probs_Y = torch.softmax(model(test_batch, feature_1, feature_2), 1)

sampled_Y = torch.multinomial(probs_Y, 1)
argmax_Y = torch.max(probs_Y, 1)[1].view(-1, 1)

print('Accuracy of sampled predictions on the test set: {:.4f}%'.format(
    (test_Y == sampled_Y.float()).sum().item() / len(test_Y) * 100))
print('Accuracy of argmax predictions on the test set: {:4f}%'.format(
    (test_Y == argmax_Y.float()).sum().item() / len(test_Y) * 100))


Result:

Accuracy of sampled predictions on the test set: 88.8889%

Accuracy of argmax predictions on the test set: 97.777778%


Reading the pytorch docs it looks like multinomial is sampling according to some distribution - just not sure how that is relevant in assessing accuracy.

I've noticed that the multinomial is non-deterministic - meaning that it is outputting a different accuracy, presumably by including different samples, each time it runs.

Here, with multinomial - we're sampling the classes with the multinomial distribution.

From the Wikipedia example,


  Suppose that in a three-way election for a large country, candidate A
  received 20% of the votes, candidate B received 30% of the votes, and
  candidate C received 50% of the votes. If six voters are selected
  randomly, what is the probability that there will be exactly one
  supporter for candidate A, two supporters for candidate B and three
  supporters for candidate C in the sample?
  
  Note: Since we’re assuming that the voting population is large, it is
  reasonable and permissible to think of the probabilities as unchanging
  once a voter is selected for the sample. Technically speaking this is
  sampling without replacement, so the correct distribution is the
  multivariate hypergeometric distribution, but the distributions
  converge as the population grows large.


If we look closely, we're doing the same, sampling without replacement.

torch.multinomial(input, num_samples, replacement=False, *, generator=None, out=None)

ref: https://pytorch.org/docs/stable/torch.html?highlight=torch%20multinomial#torch.multinomial

A multinomial experiment is a statistical experiment and it consists of n repeated trials. Each trial has a discrete number of possible outcomes. On any given trial, the probability that a particular outcome will occur is constant (that was the initial assumption).



So, the voting count can be simulated with the probabilities. Classification from soft outputs is a similar voting process.

If we repeat the experiment enough times, we'll reach as close as the actual probability.

For example, let's start with an initial probs = [0.1, 0.1, 0.3, 0.5].

We can repeat the experiment n times and count how many times an index was selected by torch.multinomial.

import torch

cnt = [0, 0, 0, 0]
for _ in range(5000):
  sampled_Y = torch.multinomial(torch.tensor([0.1, 0.1, 0.3, 0.5]), 1)
  cnt[sampled_Y[0]] += 1

print(cnt)


After 50 iterations: [6, 3, 14, 27]

After 5000 iterations: [480, 486, 1525, 2509]

After 50000 iterations: [4988, 4967, 15062, 24983]

But, this is avoided in model evaluation since it's not deterministic and requires a random generator to simulate the experiment. This is particularly useful for monte-carlo simulations, prior-posterior calculation. I have seen a graph classification example, where such evaluation was used. But I think it's not common and (even useful) in most classification tasks in machine learning.
