codelion commited on
Commit
1321a21
·
verified ·
1 Parent(s): e596b75

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -104
README.md CHANGED
@@ -37,71 +37,6 @@ A flexible, adaptive classification system that allows for dynamic addition of n
37
  pip install adaptive-classifier
38
  ```
39
 
40
- ## Quick Start
41
-
42
- ```python
43
- from adaptive_classifier import AdaptiveClassifier
44
-
45
- # Initialize with any HuggingFace model
46
- classifier = AdaptiveClassifier("bert-base-uncased")
47
-
48
- # Add some examples
49
- texts = [
50
- "The product works great!",
51
- "Terrible experience",
52
- "Neutral about this purchase"
53
- ]
54
- labels = ["positive", "negative", "neutral"]
55
-
56
- classifier.add_examples(texts, labels)
57
-
58
- # Make predictions
59
- predictions = classifier.predict("This is amazing!")
60
- print(predictions) # [('positive', 0.85), ('neutral', 0.12), ('negative', 0.03)]
61
-
62
- # Save the classifier
63
- classifier.save("./my_classifier")
64
-
65
- # Load it later
66
- loaded_classifier = AdaptiveClassifier.load("./my_classifier")
67
-
68
- # The library is also integrated with Hugging Face. So you can push and load from HF Hub.
69
-
70
- # Save to Hub
71
- classifier.push_to_hub("adaptive-classifier/model-name")
72
-
73
- # Load from Hub
74
- classifier = AdaptiveClassifier.from_pretrained("adaptive-classifier/model-name")
75
- ```
76
-
77
- ## Advanced Usage
78
-
79
- ### Adding New Classes Dynamically
80
-
81
- ```python
82
- # Add a completely new class
83
- new_texts = [
84
- "Error code 404 appeared",
85
- "System crashed after update"
86
- ]
87
- new_labels = ["technical"] * 2
88
-
89
- classifier.add_examples(new_texts, new_labels)
90
- ```
91
-
92
- ### Continuous Learning
93
-
94
- ```python
95
- # Add more examples to existing classes
96
- more_examples = [
97
- "Best purchase ever!",
98
- "Highly recommend this"
99
- ]
100
- more_labels = ["positive"] * 2
101
-
102
- classifier.add_examples(more_examples, more_labels)
103
- ```
104
-
105
  ## How It Works
106
 
107
  The system combines three key components:
@@ -112,45 +47,6 @@ The system combines three key components:
112
 
113
  3. **Adaptive Neural Layer**: Learns refined decision boundaries through continuous training
114
 
115
- ## Requirements
116
-
117
- - Python ≥ 3.8
118
- - PyTorch ≥ 2.0
119
- - transformers ≥ 4.30.0
120
- - safetensors ≥ 0.3.1
121
- - faiss-cpu ≥ 1.7.4 (or faiss-gpu for GPU support)
122
-
123
- ## Benefits of Adaptive Classification in LLM Routing
124
-
125
- We evaluate the effectiveness of adaptive classification in optimizing LLM routing decisions. Using the arena-hard-auto-v0.1 dataset with 500 queries, we compared routing performance with and without adaptation while maintaining consistent overall success rates.
126
-
127
- ### Key Results
128
-
129
- | Metric | Without Adaptation | With Adaptation | Impact |
130
- |--------|-------------------|-----------------|---------|
131
- | High Model Routes | 113 (22.6%) | 98 (19.6%) | 0.87x |
132
- | Low Model Routes | 387 (77.4%) | 402 (80.4%) | 1.04x |
133
- | High Model Success Rate | 40.71% | 29.59% | 0.73x |
134
- | Low Model Success Rate | 16.54% | 20.15% | 1.22x |
135
- | Overall Success Rate | 22.00% | 22.00% | 1.00x |
136
- | Cost Savings* | 25.60% | 32.40% | 1.27x |
137
-
138
- *Cost savings calculation assumes high-cost model is 2x the cost of low-cost model
139
-
140
- ### Analysis
141
-
142
- The results highlight several key benefits of adaptive classification:
143
-
144
- 1. **Improved Cost Efficiency**: While maintaining the same overall success rate (22%), the adaptive classifier achieved 32.40% cost savings compared to 25.60% without adaptation - a relative improvement of 1.27x in cost efficiency.
145
-
146
- 2. **Better Resource Utilization**: The adaptive system routed more queries to the low-cost model (402 vs 387) while reducing high-cost model usage (98 vs 113), demonstrating better resource allocation.
147
-
148
- 3. **Learning from Experience**: Through adaptation, the system improved the success rate of low-model routes from 16.54% to 20.15% (1.22x increase), showing effective learning from successful cases.
149
-
150
- 4. **ROI on Adaptation**: The system adapted to 110 new examples during evaluation, leading to a 6.80% improvement in cost savings while maintaining quality - demonstrating significant return on the adaptation investment.
151
-
152
- This real-world evaluation demonstrates that adaptive classification can significantly improve cost efficiency in LLM routing without compromising overall performance.
153
-
154
  ## References
155
 
156
  - [RouteLLM: Learning to Route LLMs with Preference Data](https://arxiv.org/abs/2406.18665)
 
37
  pip install adaptive-classifier
38
  ```
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ## How It Works
41
 
42
  The system combines three key components:
 
47
 
48
  3. **Adaptive Neural Layer**: Learns refined decision boundaries through continuous training
49
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  ## References
51
 
52
  - [RouteLLM: Learning to Route LLMs with Preference Data](https://arxiv.org/abs/2406.18665)