SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 512 tokens
Number of Classes: 10 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
Disclaimer	'"Terms of Use API restrictions?"' '"Terms of Use age restrictions?"' '"Disclaimer update alerts?"'
IB Loan against Sovereign Gold Bond	'"Sovereign Jewel Bond loan margin requirements?"' '"Sovereign Gold Bond joint holder rules?"' '"Sovereign Jewel Bond nomination process?"'
Ind Advantage (Reward Program)	'"Advantage Rewards international redemption fees?"' '"Blackout dates for reward travel bookings?"' '"Advantage Program customer support channels?"'
Amalgamation	'"Merger documentation checklist for branches?"' '"Banking Amalgamation customer notification process?"' '"Amalgamation loan portfolio transfer details?"'
Loan / OD against NSC / KVP / Relief bonds of RBI / LIC policies	'"Relief Bonds OD interest payment frequency?"' '"KVP valuation for overdraft approval criteria?"' '"NSC loan documentation checklist?"'
Chief General Managers	'"Chief General Managers' office working hours?"' '"How to contact Chief General Managers for escalations?"' '"Senior General Managers' regional jurisdiction list?"'
Point of Sale (PoS)	'"Offline PoS transaction capabilities?"' '"PoS transaction audit trails?"' '"PoS batch settlement timing?"'
Featured Products / Services / Schemes	'"Highlighted Products insurance coverage details?"' '"Highlighted Products loan-to-value ratio?"' '"Featured schemes disbursement timeline?"'
e-Allahabad Bank Journey	'"e-Allahabad Experience customer support channels?"' '"Allahabad Online Journey QR code payments?"' '"Allahabad Online Journey statement download process?"'
Centralized Pension Processing Centre	'"Processing time for pension applications?"' '"QR code payments at Payment Office?"' '"Central Pension Management Centre contact details?"'

Evaluation

Metrics

Label	Accuracy
all	0.975

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("kneau007/my-classifier")
# Run inference
preds = model("\"Relief Bonds redemption during OD tenure?\"")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	5.2062	8

Label	Training Sample Count
Amalgamation	14
Chief General Managers	16
Disclaimer	11
Featured Products / Services / Schemes	18
IB Loan against Sovereign Gold Bond	18
Ind Advantage (Reward Program)	19
Loan / OD against NSC / KVP / Relief bonds of RBI / LIC policies	16
Point of Sale (PoS)	16
e-Allahabad Bank Journey	15
Centralized Pension Processing Centre	17

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0025	1	0.172	-
0.125	50	0.1198	-
0.25	100	0.0251	-
0.375	150	0.0068	-
0.5	200	0.003	-
0.625	250	0.0018	-
0.75	300	0.0015	-
0.875	350	0.0013	-
1.0	400	0.0013	-

Framework Versions

Python: 3.11.11
SetFit: 1.1.1
Sentence Transformers: 3.4.1
Transformers: 4.48.3
PyTorch: 2.5.1+cu124
Datasets: 3.3.2
Tokenizers: 0.21.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

kneau007
/

my-classifier