Is cross-entropy a good cost function if I'm interested in the probabilities of a sample belonging to a certain class?
Cross entropy is indeed appropriate for multiclass classification.

When the tensorflow documentation states about cross_entropy that:


  "Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class)."


it is referring to the fact that, at a conceptual level, classes are not expected to overlap. An example of non-overlapping classes are "cat" and "dog". An example of overlapping classes are "Elephant" and "African Elephant"; this type of problem is called multilabel classification, because instead of classes, you have labels and each individual can be assigned many labels.

Update: with the new information about the problem, we can tell that the problem being faced is not a classification one, as the desired output are specific probability values. This implies that the problem is a regression one. As the outputs are probabilities, it is appropriate to use softmax to ensure they add up to 1. As it is regression, it would be appropriate to use MSE as loss function.