Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,87 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
metrics:
|
6 |
+
- accuracy
|
7 |
+
base_model:
|
8 |
+
- microsoft/deberta-v3-small
|
9 |
+
pipeline_tag: text-classification
|
10 |
+
---
|
11 |
+
# A1-DeBERTaV3-Small
|
12 |
+
|
13 |
+
## Model Description
|
14 |
+
|
15 |
+
A1-DeBERTaV3-Small is a hybrid model that combines the [DeBERTa v3 small](https://huggingface.co/microsoft/deberta-v3-small) encoder with a transformer based sentiment classifier. The DeBERTa v3 component receives tokenized text and outputs last hidden states for each token. These embeddings are then passed to Waveform A1, which aggregates the token representations, applies multi-head self-attention and a position-wise feed-forward network, and finally produces joint predictions for both sentiment and market classification.
|
16 |
+
|
17 |
+
## Intended Use Cases
|
18 |
+
|
19 |
+
- **Crypto Discussion Analysis**: Automatic categorization of large volumes of messages into sentiment and market outlook.
|
20 |
+
- **Real-Time Monitoring**: Scalable pipeline for near-real-time classification of crypto-related chatter.
|
21 |
+
- **Research & Development**: A testbed for exploring semi-supervised or domain-specific language modeling strategies.
|
22 |
+
|
23 |
+
|
24 |
+
## Example Usage
|
25 |
+
|
26 |
+
```python
|
27 |
+
from transformers import AutoTokenizer
|
28 |
+
import onnxruntime
|
29 |
+
import numpy as np
|
30 |
+
import torch.nn.functional as F
|
31 |
+
|
32 |
+
def decode_sentiment(idx: int) -> str:
|
33 |
+
sentiment_map = {0: 'positive', 1: 'neutral', 2: 'negative'}
|
34 |
+
return sentiment_map[idx]
|
35 |
+
|
36 |
+
def decode_market(idx: int) -> str:
|
37 |
+
market_map = {
|
38 |
+
0: 'strong bullish',
|
39 |
+
1: 'bullish',
|
40 |
+
2: 'neutral',
|
41 |
+
3: 'bearish',
|
42 |
+
4: 'strong bearish'
|
43 |
+
}
|
44 |
+
return market_map[idx]
|
45 |
+
|
46 |
+
def softmax(x, axis=1):
|
47 |
+
exp_x = np.exp(x - np.max(x, axis=axis, keepdims=True))
|
48 |
+
return exp_x / np.sum(exp_x, axis=axis, keepdims=True)
|
49 |
+
|
50 |
+
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-v3-small")
|
51 |
+
|
52 |
+
text = "input-text-goes-here"
|
53 |
+
|
54 |
+
inputs = tokenizer(
|
55 |
+
text,
|
56 |
+
return_tensors="pt",
|
57 |
+
padding="max_length",
|
58 |
+
truncation=True,
|
59 |
+
max_length=512
|
60 |
+
)
|
61 |
+
input_ids = inputs["input_ids"]
|
62 |
+
attention_mask = inputs["attention_mask"]
|
63 |
+
|
64 |
+
ort_inputs = {
|
65 |
+
"input_ids": input_ids.cpu().numpy(),
|
66 |
+
"attention_mask": attention_mask.cpu().numpy()
|
67 |
+
}
|
68 |
+
|
69 |
+
session = onnxruntime.InferenceSession("a1-debertav3.onnx")
|
70 |
+
|
71 |
+
sentiment_logits, market_logits = session.run(None, ort_inputs)
|
72 |
+
|
73 |
+
sentiment_probs = softmax(sentiment_logits, axis=1)
|
74 |
+
market_probs = softmax(market_logits, axis=1)
|
75 |
+
|
76 |
+
sentiment_pred = np.argmax(sentiment_probs, axis=1)
|
77 |
+
market_pred = np.argmax(market_probs, axis=1)
|
78 |
+
|
79 |
+
decoded_sentiment = decode_sentiment(sentiment_pred.item())
|
80 |
+
decoded_market = decode_market(market_pred.item())
|
81 |
+
print(f"Sentiment: {decoded_sentiment}")
|
82 |
+
print(f"Market: {decoded_market}")
|
83 |
+
|
84 |
+
```
|
85 |
+
|
86 |
+
## Community
|
87 |
+
This model is actively maintained and open to community contributions via pull requests or collaboration inquiries.
|