Where to apply Montgomery Multiplication in GF(2^n) I'm optimizing a Reed Solomon decoding library for several polynomials in $\operatorname{GF}(2^k)$, $k\in\{8,10,12\}$.
Reading about the Montgomery Multiplication from Çetin K. Koç &amp; Tolga Acar's Montgomery Multiplication in $\operatorname{GF}(2^k)$ (in Designs, Codes and Cryptography, 1998), I'm wondering if and how the method could help in polynomial reduction after carry free multiplication.

$t(x) = a(x)b(x)$ is a full length (2k-1) polynomial from carryless multiplication
$u(x) = t(x)n'(x) \bmod r(x)$  returns lower half of the $\operatorname{GF}$ multiplication
$(u(x)n(x) + t(x)) / r(x)$ corresponds to the higher half of one $\operatorname{GF}$ multiplication and one XOR

Instead of calculating $a(x)b(x) \bmod n(x)$, I have now $a(x)b(x)r^{-1}(x) \bmod n(x)$, and there's just one place where I can imagine using it. In inversion by modular exponentiation, where instead of calculating $a(x)^{254}$, I can calculate $(a(x)r^{-1}(x))^{254}$ and multiply that with a constant $r^{254}(x)$.
But can it be used e.g. in polynomial evaluation (used e.g. in syndrome calculation) or for modular reduction after $\operatorname{GF}$ multiplication?
The usual motivation for using Montgomery multiplication is that it significantly reduces the cost of modular reduction by changing the representation of elements. In the Montgomery ring the polynomial $A(x)$ is instead represented by $a(x)=A(x)r(x)\pmod{n(x)}$ where $r(x)$ is a fixed element of the same degree as $n(x)$, but for which division is very efficient (e.g. $x^k$). If likewise we have $B(x)$ represented by $b(x)=B(x)r(x)$ then $a(x)b(x)r^{-1}(x)\equiv A(x)B(x)r(x)\pmod{n(x)}$ which is the representation that we would choose for $A(x)B(x)$ and so your algorithm is way of implementing modular multiplication of polynomials in this representation without having to do a modular reduction. Addition and subtraction do not need new algorithms. If you have an algorithm involving many modular multiplies, it can be worthwhile converting to Montgomery ring representation, using Montgomery arithmetic and then converting back.
To convert $a(x)$ back to $A(x)$ we need to multiply by $r^{-1}(x)$, so it can be useful to precompute and save $r^{-1}(x)\pmod{n(x)}$. I don't know of any way to avoid modular reduction in this final conversion.
Evaluating using the Montgomery representation is possible, but requires a little extra work: to evaluate $A(x)\pmod{n(x)}$ given its representation $a(x)$ we have to evaluate $a(x)\pmod{n(x)}$ and $r^{-1}(x)\pmod{n(x)}$ and multiply the two answers together. However, if many evaluations are needed for the same input $m$, the value $r^{-1}(m)$ can be reused and the overhead is not as great.