Mini-GTE

Overview

This is the first model developed by QTACK and serves as a proof of concept for our distillation approach! Built upon a distillbert-based architecture, Mini-GTE is distilled from GTE and designed for efficiency without sacrificing accuracy at only 66M parameters. As a standalone sentence transformer, it ranks 2nd on the MTEB classic leaderboard in the <100M parameter category and 63rd overall which makes it a strong choice for real-time query encoding, semantic search, and similarity tasks.

Model Details

• Model Type: Sentence Transformer
• Base model: distilbert/distilbert-base-uncased
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Usage

• Optimized for quick inference
• Great at quickly generating high quality encodings
• Easy to plug and play since it is distilled from GTE
• We want to see how you’re using our model so we’ll give you a free coffee/$10 gift card if you get on call with us and show us what you’ve built!

Getting Started

Installation

Mini-GTE is built on the Sentence Transformers framework. To install the required packages, run:

pip install -U sentence-transformers

Quick Start

Here's a quick example to get you started:

from sentence_transformers import SentenceTransformer

# Download directly from Hugging Face
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'The weather is lovely today.',
    "It's so sunny outside!",
    'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape) # Expected: [3, 768]

# Compute the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape) # Expected: [3, 3]

Training Details

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.48.0.dev0
• PyTorch: 2.1.0a0+32f93b1
• Accelerate: 1.2.0
• Datasets: 2.21.0
• Tokenizers: 0.21.0

Getting Help

For any questions, suggestions, or issues, please contact the QTACK team directly through our contact page.