Is function Deterministic or Probabilistic in the malicious security definition? As you see, according to the attached definitions from "Efficient Secure Two-Party Protocols", the definition of the security in the semi-honest world has two cases for $f=(f_1,f_2)$.(deterministic and probabilistic)



Now, my question is, why the authors have not said anything about the type of the $f$ in the malicious world? I mean, whether $f$ has two types like in the semi-honest model or not?
No, the semi-honest definition does not have two cases; the correct definition is Definition 2.2.1, which applies to all functionalities. But if the functionality is deterministic, then the simpler definition given later in the book (yes, it is given later in the book; I am not sure why you decided to put it above it in your question) is equivalent, and might therefore be preferred when trying to prove the security of a protocol.

No similar "simplified" definition exists in the malicious model, because even if the functionality is deterministic and the protocol correctly computes it, a malicious adversary can deviate from the protocol and output something completely different from what the protocol says it should.