What is the advantage of designing deterministic distributed algorithms?
I will answer this from the perspective of distributed graph algorithms (distributed algorithms that solve a graph problem related to the structure of the communication network).

Here are some non-obvious reasons for designing deterministic distributed algorithms in this setting:


Subroutines in randomised algorithms. On p. 12–13 of these slides, Elkin outlines an algorithm design technique in which you can use a fast deterministic distributed algorithms as a subroutine in order to construct a fast randomised distributed algorithm. Interestingly, it is not possible to use a typical randomised algorithm as a subroutine in the same context (the error probability would be too high).
Fault tolerance. There is a mechanical translation that allows you to convert a fast deterministic distributed algorithm into a fast self-stabilising distributed algorithm (see e.g. Section 2.4 of this survey). A similar conversion is not known for randomised algorithms (and I think it is unlikely to exist in the general case).