tzoulio commited on
Commit
f34efe3
·
verified ·
1 Parent(s): 8a77f3c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md CHANGED
@@ -77,3 +77,74 @@ pipeline = transformers.pipeline(
77
  text = "Insert the news article text here..."
78
  prediction = pipeline(text)
79
  print(prediction)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  text = "Insert the news article text here..."
78
  prediction = pipeline(text)
79
  print(prediction)
80
+ ```
81
+ ### **Example Prediction Output**
82
+
83
+ ```python
84
+ [
85
+ {
86
+ "label": "LABEL_2",
87
+ "score": 0.47
88
+ }
89
+ ]
90
+
91
+ LABEL_0: Left
92
+ LABEL_1: Center
93
+ LABEL_2: Right
94
+ ```
95
+
96
+ ### **Input / Output Details**
97
+
98
+ **Input**: A single string containing the text of a news article.
99
+ **Output**: A list of dictionaries, where each dictionary contains:
100
+ - "label": The predicted label (e.g., "LABEL_2")
101
+ - "score": The probability for that label.
102
+ ```css
103
+ Example Output: [[{"LABEL_0": 0.23, "LABEL_1": 0.30, "LABEL_2": 0.47}]]
104
+ Indicates 23% chance of Left, 30% chance of Center, 47% chance of Right.
105
+
106
+ ```
107
+
108
+ ## **Training & Fine-tuning**
109
+ ### **Dataset Sizes**
110
+
111
+ - **Training Set**: 17,984 examples
112
+ - **Evaluation Set**: 4,496 examples
113
+ - **Test Set**: 5,620 examples
114
+
115
+ ### **Hyperparameters and Important Settings**
116
+ ```python
117
+ # Precision & Quantization
118
+ load_in_4bit = True
119
+ bnb_4bit_use_double_quant = True
120
+ bnb_4bit_quant_type = "nf4"
121
+ bnb_4bit_compute_dtype = torch.bfloat16
122
+
123
+ # LoRA Configuration
124
+ lora_r = 16
125
+ lora_alpha = 64
126
+ lora_dropout = 0.1
127
+ bias = "none"
128
+
129
+ # Task Type
130
+ task_type = "SEQ_CLS"
131
+
132
+ # Training Setup
133
+ per_device_train_batch_size = 4
134
+ gradient_accumulation_steps = 4
135
+ learning_rate = 2e-4
136
+ optim = "paged_adamw_32bit"
137
+ num_train_epochs = 3
138
+ warmup_steps = 2
139
+ fp16 = True
140
+ logging_steps = 1
141
+ ```
142
+ ## **Evaluation**
143
+ ### **Metrics**
144
+ We report the F1-score on each dataset split.
145
+ ## **Results**
146
+
147
+ - F1-Score (Training): 0.96658
148
+ - F1-Score (Eval): 0.96664
149
+ - F1-Score (Test): 0.96299
150
+