Do I need to encode samples during inference? I recently started saving (pickling) my fitted encoders. The thinking was that I would need them to encode previously unseen samples during inference.

Encode training features and labels.
Train model on encoded data.
Export fitted encoders alongside model.
New/ unseen data arrives.
Encode features of new data.
Run model prediction.
Decode prediction output.

Is this mandatory?
Yes the encoders need to be rerun during inference time. You basically need to run the exact same data preprocessing steps prior to training on the unseen observations. Without them your model will not be to recognize unseen data and hence fair poorly.