Why is sampling useful in machine learning?
TL:DR - If you know the posterior distribution of the complex process (i.e. the output distribution), and that distribution is one which can be modelled with reasonable accuracy, then sampling from it should reasonably represent the responses of the complex system.



An example could be a complex decision process, whereby many decisions are made consecutively, maybe with some conditional or temporal relationships along the way (basically any process that is considered to be complex). Now imagine, after all this complexity, there are e.g. two possible categorically outputs: a zero or a one. Well this something we might be able to model with a Bernoulli distribution, assuming we can estimate a reasonable parameter $p$ (specific to the Bernoulli distribution).

By sampling from the distribution, we would hope to draw samples, which are representative of the complex process. I hope this covers your main question.
I think point 4. is also clear to understand as well from my trivial example. I am unsure as to what cost refers exactly in point 3. 

As for point 1., there are examples where (random) sampling, for example in a grid search of model/optimisation parameters can improve results, as it results in an improved exploration of the parameter space compared to other methods such as grid-search.