Pytorch RNN with no nonlinearity
Is it possible to implement an RNN layer with no nonlinearity in Pytorch like in Keras where one can set the activation to linear? By removing the nonlinearlity, I want to implement a first-order infinite-impulse-response (IIR) filter with a differentiable parameter and integrate it into my model for end-to-end learning. I can obviously implement the filter in Pytorch but I thought using an inbuilt function may be more efficient.

Removing non-linearity from RNN turns it into a linear dense layer without any activation.
If that is what you want, then simply use nn.linear and set activation to None
Explanation
Here is why this happens. Fundamentally, an RNN for timesteps works as below -

h(t) = tanh(U.x(t) + W.h(t−1) + b)

h(0) = tanh(U0.x(0) + b0)
h(1) = tanh(U1.x(1) + W1.h(0) + b1)
h(2) = tanh(U2.x(2) + W2.h(1) + b2)

#... and so on.

If you remove linearity by removing the tanh, here is what happens -
h(0) = U0.x(0) + b0
h(1) = U1.x(1) + W1.h(0) + b1
     = U1.x(1) + W1.(U0.x(0) + b0) + b1 #expanding x(0)
     = U1.x(1) + W1.U0.x(0) + W1.b0 + b1
     = U1.x(1) + W1.U0.x(0) + W1.b0 + b1
     = V1.x(1) + V0.x(0) + C    #Can be rewritten with new weights
     = V . x + C    #General form

So the final form of the state of an RNN after 2 timesteps is simply Wx+b like the linear layer without activation.
In other words, removing the non-linearity from an RNN turns it into a linear dense layer without any activation, completely removing the notion of time-steps.
