Numpy random.choice probablities don't sum up to 1
I wanna randomly select sample points based on the probability distribution specified by prob for a given row. However, I get the error in np.random.choice that the probabilities don't add up to 1. This is very weird because I first normalize using the L1-norm along the rows and then I define a uniform distribution if the values are smaller than the threshold 1e-6.

import numpy as np
import torch.nn.functional as F

prob = F.normalize(outputs, p=1, dim=1).clone().data.cpu().numpy() # outputs is a torch.Tensor of shape (14, 6890)
all_zero = np.where(prob.max(1) &lt; 1e-6)[0] # find indices of rows where all values are smaller
prob[all_zero] = np.full(prob.shape[1], 1 / prob.shape[1]) # fill those rows uniformly
# ... somewhere later inside a method
for j in range(14):
    sample = np.random.choice(6890, 4, replace=False, p=prob[j])


Do you understand, why that is?

As error suggests prob[j] doesn't sum to 1. 

Your epsilon 1e-6 is way too big to be considered insignificant, there is no need for this operation at all. If you insist, you have to redistribute zero-ed out values across what's left to 1 (and it seems you did just that actually).

All in all you didn't normalize the array to 1:

prob /= prob.sum(axis=1) # make it prob dist


BTW. Broadcasting will extend your single number to the whole row, no need for np.full:

prob[all_zero] = 1 / prob.shape[1]

