How to get the probability/closeness of a sample belonging to a specific cluster?
Try Gaussian Mixture Model (GMM) as it is similar to KMeans, but differs in a few ways. In a nutshell think of KMeans as a hard clustering model where each sample is assigned to only one cluster, whereas GMM is a soft clustering technique that calculates the density (probability) of each Gaussian mixture (which can be considered as clusters) containing the data point in question. You can get both labels and probability scores from the model. Try it and see if it helps in your case. It is available from the SciKit Learn library.
Another approach in case if you have to stick to KMeans could be:

Take the cluster centers from the KMeans model.
Take your test sample vector and pass these as parameters to the softmax function to get the probability score for all the cluster centers per sample.