title: Using the encoder part only from T5 model
body: <p>I want to build a classification model that needs only the encoder part of language models. I have tried Bert, Roberta, xlnet, and so far I have been successful.</p>
  <p>I now want to test the encoder part only from T5, so far, I found encT5 <a href="https://github.com/monologg/EncT5" rel="nofollow noreferrer">https://github.com/monologg/EncT5</a></p>
  <p>And T5EncoderModel from HuggingFace.</p>
  <p>Can anyone help me understand if T5EncoderModel is what I am looking for or not?</p>
  <p>It says in the description: The bare T5 Model transformer outputting encoder’s raw hidden-states without any specific head on top.</p>
  <p>This is slightly confusing to me, especially that encT5 mentioned that they implemented the encoder part only because it didn't exist in HuggingFace which is what makes me more doubtful here.</p>
  <p>Please note that I am a beginner in deep learning, so please go easy on me I understand that ny questions can be naive to most of you.</p>
  <p>Thank you</p>