Update README.md
Browse files
README.md
CHANGED
@@ -7,6 +7,10 @@ tags:
|
|
7 |
model-index:
|
8 |
- name: ModernBERT-base-mask-finetuned-shakespeare
|
9 |
results: []
|
|
|
|
|
|
|
|
|
10 |
---
|
11 |
|
12 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
@@ -18,17 +22,72 @@ This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://hugg
|
|
18 |
It achieves the following results on the evaluation set:
|
19 |
- Loss: 2.2340
|
20 |
|
21 |
-
##
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
26 |
-
|
27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
|
29 |
## Training and evaluation data
|
30 |
|
31 |
-
|
32 |
|
33 |
## Training procedure
|
34 |
|
@@ -57,4 +116,4 @@ The following hyperparameters were used during training:
|
|
57 |
- Transformers 4.48.3
|
58 |
- Pytorch 2.5.1+cu124
|
59 |
- Datasets 3.3.2
|
60 |
-
- Tokenizers 0.21.0
|
|
|
7 |
model-index:
|
8 |
- name: ModernBERT-base-mask-finetuned-shakespeare
|
9 |
results: []
|
10 |
+
datasets:
|
11 |
+
- 2nji/Shakespeare_Corpus
|
12 |
+
language:
|
13 |
+
- en
|
14 |
---
|
15 |
|
16 |
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
|
|
22 |
It achieves the following results on the evaluation set:
|
23 |
- Loss: 2.2340
|
24 |
|
25 |
+
## How to use
|
26 |
+
|
27 |
+
You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
|
28 |
+
|
29 |
+
```python
|
30 |
+
import torch
|
31 |
+
from transformers import pipeline
|
32 |
+
from pprint import pprint
|
33 |
+
|
34 |
+
pipe = pipeline(
|
35 |
+
"fill-mask",
|
36 |
+
model="2nji/ModernBERT-base-mask-finetuned-shakespeare",
|
37 |
+
torch_dtype=torch.bfloat16,
|
38 |
+
)
|
39 |
+
|
40 |
+
input_text = "Thou [MASK] on [MASK]."
|
41 |
+
results = pipe(input_text)
|
42 |
+
pprint(results)
|
43 |
+
|
44 |
+
|
45 |
+
<!-- [[{'score': 0.71875,
|
46 |
+
'sequence': '[CLS]Thou art on[MASK].[SEP]',
|
47 |
+
'token': 1445,
|
48 |
+
'token_str': ' art'},
|
49 |
+
{'score': 0.1416015625,
|
50 |
+
'sequence': '[CLS]Thou hast on[MASK].[SEP]',
|
51 |
+
'token': 16579,
|
52 |
+
'token_str': ' hast'},
|
53 |
+
{'score': 0.014892578125,
|
54 |
+
'sequence': '[CLS]Thou be on[MASK].[SEP]',
|
55 |
+
'token': 320,
|
56 |
+
'token_str': ' be'},
|
57 |
+
{'score': 0.00701904296875,
|
58 |
+
'sequence': '[CLS]Thou Art on[MASK].[SEP]',
|
59 |
+
'token': 3975,
|
60 |
+
'token_str': ' Art'},
|
61 |
+
{'score': 0.0042724609375,
|
62 |
+
'sequence': '[CLS]Thou call on[MASK].[SEP]',
|
63 |
+
'token': 1067,
|
64 |
+
'token_str': ' call'}],
|
65 |
+
[{'score': 0.1767578125,
|
66 |
+
'sequence': "[CLS]Thou[MASK] on't.[SEP]",
|
67 |
+
'token': 626,
|
68 |
+
'token_str': "'t"},
|
69 |
+
{'score': 0.146484375,
|
70 |
+
'sequence': '[CLS]Thou[MASK] on me.[SEP]',
|
71 |
+
'token': 479,
|
72 |
+
'token_str': ' me'},
|
73 |
+
{'score': 0.0419921875,
|
74 |
+
'sequence': '[CLS]Thou[MASK] on it.[SEP]',
|
75 |
+
'token': 352,
|
76 |
+
'token_str': ' it'},
|
77 |
+
{'score': 0.0419921875,
|
78 |
+
'sequence': '[CLS]Thou[MASK] on earth.[SEP]',
|
79 |
+
'token': 6149,
|
80 |
+
'token_str': ' earth'},
|
81 |
+
{'score': 0.03955078125,
|
82 |
+
'sequence': '[CLS]Thou[MASK] on him.[SEP]',
|
83 |
+
'token': 779,
|
84 |
+
'token_str': ' him'}]] -->
|
85 |
+
|
86 |
+
```
|
87 |
|
88 |
## Training and evaluation data
|
89 |
|
90 |
+
This model was finetuned using the the [Shakespare_corpus](https://huggingface.co/datasets/2nji/Shakespeare_Corpus) Dataset
|
91 |
|
92 |
## Training procedure
|
93 |
|
|
|
116 |
- Transformers 4.48.3
|
117 |
- Pytorch 2.5.1+cu124
|
118 |
- Datasets 3.3.2
|
119 |
+
- Tokenizers 0.21.0
|