ONNX
English
Shing Yee
feat: add files
1d52dd2 unverified
|
raw
history blame
1.89 kB
metadata
license: other
license_name: govtech-singapore
license_link: LICENSE

Off-Topic Classification Model

Off-Topic Classification Model

This model leverages a fine-tuned Cross Encoder STSB Roberta Base to perform binary classification, determining whether a user prompt is off-topic in relation to the system's intended purpose as defined by the system prompt.

Model Highlights

  • Base Model: stsb-roberta-base
  • Maximum Context Length: 1024 tokens
  • Task: Binary classification (on-topic/off-topic)

Performance

Approach Model ROC-AUC F1 Precision Recall
Fine-tuned cross-encoder classifier stsb-roberta-base 0.99 0.99 0.99 0.99
Pre-trained cross-encoder stsb-roberta-base 0.73 0.68 0.53 0.93

Usage

  1. Clone this repository and install the required dependencies:

    pip install -r requirements.txt
    
  2. You can run the model using two options:

    Option 1: Using inference_onnx.py with the ONNX Model.

     ```
     python inference_onnx.py '[
         ["System prompt example 1", "User prompt example 1"],
         ["System prompt example 2", "System prompt example 2]
     ]'
     ```
    

    Option 2: Using inference_safetensors.py with PyTorch and SafeTensors.

     ```
     python inference_safetensors.py '[
         ["System prompt example 1", "User prompt example 1"],
         ["System prompt example 2", "System prompt example 2]
     ]'
     ```
    

Read more about this model in our technical report.