How are mutated transactions prevented from getting relayed?
Transaction malleability is prevented by selecting one canonical representation of every transaction and making every other representation nonstandard. That is, of all possible transactions one transaction can be mutated into, exactly one is considered "original", and other representations are not relayed.

Originally, Bitcoin could accept different representations of the same data. For example, it were possible to represent the same number both with and without leading zeros, producing equivalent transactions with different binary representations. Now, a transaction would only be considered standard if those numbers are represented using a specific number of digits, so it is now impossible to write the same number in different ways.

However, this rule is not enforced on transactions that are already in a block (since otherwise it would be a fork), so a powerful adversary can still push a mutated transaction into the network by mining it himself.