2nji commited on
Commit
b8d6931
·
verified ·
1 Parent(s): 504f6dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +68 -9
README.md CHANGED
@@ -7,6 +7,10 @@ tags:
7
  model-index:
8
  - name: ModernBERT-base-mask-finetuned-shakespeare
9
  results: []
 
 
 
 
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -18,17 +22,72 @@ This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://hugg
18
  It achieves the following results on the evaluation set:
19
  - Loss: 2.2340
20
 
21
- ## Model description
22
-
23
- More information needed
24
-
25
- ## Intended uses & limitations
26
-
27
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Training and evaluation data
30
 
31
- More information needed
32
 
33
  ## Training procedure
34
 
@@ -57,4 +116,4 @@ The following hyperparameters were used during training:
57
  - Transformers 4.48.3
58
  - Pytorch 2.5.1+cu124
59
  - Datasets 3.3.2
60
- - Tokenizers 0.21.0
 
7
  model-index:
8
  - name: ModernBERT-base-mask-finetuned-shakespeare
9
  results: []
10
+ datasets:
11
+ - 2nji/Shakespeare_Corpus
12
+ language:
13
+ - en
14
  ---
15
 
16
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
22
  It achieves the following results on the evaluation set:
23
  - Loss: 2.2340
24
 
25
+ ## How to use
26
+
27
+ You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
28
+
29
+ ```python
30
+ import torch
31
+ from transformers import pipeline
32
+ from pprint import pprint
33
+
34
+ pipe = pipeline(
35
+ "fill-mask",
36
+ model="2nji/ModernBERT-base-mask-finetuned-shakespeare",
37
+ torch_dtype=torch.bfloat16,
38
+ )
39
+
40
+ input_text = "Thou [MASK] on [MASK]."
41
+ results = pipe(input_text)
42
+ pprint(results)
43
+
44
+
45
+ <!-- [[{'score': 0.71875,
46
+ 'sequence': '[CLS]Thou art on[MASK].[SEP]',
47
+ 'token': 1445,
48
+ 'token_str': ' art'},
49
+ {'score': 0.1416015625,
50
+ 'sequence': '[CLS]Thou hast on[MASK].[SEP]',
51
+ 'token': 16579,
52
+ 'token_str': ' hast'},
53
+ {'score': 0.014892578125,
54
+ 'sequence': '[CLS]Thou be on[MASK].[SEP]',
55
+ 'token': 320,
56
+ 'token_str': ' be'},
57
+ {'score': 0.00701904296875,
58
+ 'sequence': '[CLS]Thou Art on[MASK].[SEP]',
59
+ 'token': 3975,
60
+ 'token_str': ' Art'},
61
+ {'score': 0.0042724609375,
62
+ 'sequence': '[CLS]Thou call on[MASK].[SEP]',
63
+ 'token': 1067,
64
+ 'token_str': ' call'}],
65
+ [{'score': 0.1767578125,
66
+ 'sequence': "[CLS]Thou[MASK] on't.[SEP]",
67
+ 'token': 626,
68
+ 'token_str': "'t"},
69
+ {'score': 0.146484375,
70
+ 'sequence': '[CLS]Thou[MASK] on me.[SEP]',
71
+ 'token': 479,
72
+ 'token_str': ' me'},
73
+ {'score': 0.0419921875,
74
+ 'sequence': '[CLS]Thou[MASK] on it.[SEP]',
75
+ 'token': 352,
76
+ 'token_str': ' it'},
77
+ {'score': 0.0419921875,
78
+ 'sequence': '[CLS]Thou[MASK] on earth.[SEP]',
79
+ 'token': 6149,
80
+ 'token_str': ' earth'},
81
+ {'score': 0.03955078125,
82
+ 'sequence': '[CLS]Thou[MASK] on him.[SEP]',
83
+ 'token': 779,
84
+ 'token_str': ' him'}]] -->
85
+
86
+ ```
87
 
88
  ## Training and evaluation data
89
 
90
+ This model was finetuned using the the [Shakespare_corpus](https://huggingface.co/datasets/2nji/Shakespeare_Corpus) Dataset
91
 
92
  ## Training procedure
93
 
 
116
  - Transformers 4.48.3
117
  - Pytorch 2.5.1+cu124
118
  - Datasets 3.3.2
119
+ - Tokenizers 0.21.0