# Collection google/sentence-t5/1

Collection of sentence T5 encoders trained on variety of data.

<!-- task: text-embedding -->
<!-- network-architecture: transformer -->
<!-- language: en -->

## Overview

The sentence-T5 family of models encode text into high-dimensional vectors that
can be used for text classification, semantic similarity, clustering and other
natural language processing tasks.

Our model is built on top of
[the Text-To-Text Transfer Transformer (T5)](https://arxiv.org/pdf/1910.10683.pdf).
It is trained on a variety of data sources and initialized from pre-trained T5
models with different model sizes as described in [1]. The input is
variable-length English text and the output is a 768-dimensional vector. Note
that there's no hard length limit for T5 (i.e., no 512 tokens limit as for
BERT), but that it's been trained to produce good embeddings for approximately
sentence length text.

#### Models

There are several versions of sentence T5 encoder models trained with different
goals including size/performance.

Model                                                                 | Comments
--------------------------------------------------------------------- | --------
[sentence-t5-base](https://tfhub.dev/google/sentence-t5/st5-base/1)   | 12-layer, 12 heads, encoder-only
[sentence-t5-large](https://tfhub.dev/google/sentence-t5/st5-large/1) | 24-layer, 16 heads, encoder-only
[sentence-t5-3b](https://tfhub.dev/google/sentence-t5/st5-3b/1)       | 24-layer, 32 heads, encoder-only

## References

[1] Jianmo, Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B. Hall,
Daniel Cer, Yinfei Yang.
[Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models.](https://arxiv.org/abs/2108.08877)
August 2021.

Add the Sentence T5 collection to TF hub.
Add the Sentence T5 collection to TF hub.

PiperOrigin-RevId: 414875374
