Hugging Face's logo

AISE-TUDelft
/
Custom-Activations-BERT-Adaptive-GELU

Fill-Mask
Transformers
PyTorch
activations_roberta
Inference Endpoints
Model card Files Community
filipignijic commited on
Commit
ca78cbb
·
verified ·
1 Parent(s): 9ae1db0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -1,9 +1,11 @@
 
1
  datasets:
2
  - roneneldan/TinyStories
3
  metrics:
4
- - babyLM
5
  ---
6
  Basemodel: roBERTa
 
7
  Configs:
8
  Vocab size: 10,000
9
  Hidden size: 512
@@ -12,12 +14,11 @@ Configs:
12
  Number of heads: 4
13
  Window size: 256
14
  Intermediate-size: 1024
15
- 57.69 [56.75, 58.73] 59.25 [58.78, 59.65]
16
 
17
  Results:
18
  - Task: glue
19
  Score: 57.69
20
  Confidence Interval: [56.75, 58.73]
21
  - Task: blimp
22
- Score: 55.64
23
- Confidence Interval: [58.78, 59.65]
 
1
+ ---
2
  datasets:
3
  - roneneldan/TinyStories
4
  metrics:
5
+ - babylm
6
  ---
7
  Basemodel: roBERTa
8
+
9
  Configs:
10
  Vocab size: 10,000
11
  Hidden size: 512
 
14
  Number of heads: 4
15
  Window size: 256
16
  Intermediate-size: 1024
 
17
 
18
  Results:
19
  - Task: glue
20
  Score: 57.69
21
  Confidence Interval: [56.75, 58.73]
22
  - Task: blimp
23
+ Score: 59.25
24
+ Confidence Interval: [58.78, 59.65]