Xenova HF Staff commited on
Commit
2715498
·
verified ·
1 Parent(s): db1fc70

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -7,6 +7,31 @@ tags: []
7
 
8
  <!-- Provide a quick summary of what the model is/does. -->
9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
 
12
  ## Model Details
 
7
 
8
  <!-- Provide a quick summary of what the model is/does. -->
9
 
10
+ ## Code to generate
11
+
12
+ ```py
13
+ from transformers import AutoConfig, LlamaForCausalLM, AutoTokenizer
14
+ import torch
15
+
16
+ model_id='meta-llama/Llama-3.2-1B-Instruct'
17
+ config = AutoConfig.from_pretrained(
18
+ model_id,
19
+ head_dim=8,
20
+ hidden_size=32,
21
+ num_attention_heads=4,
22
+ num_key_value_heads=2,
23
+ num_hidden_layers=2,
24
+ intermediate_size=64,
25
+ )
26
+
27
+ model = LlamaForCausalLM(config)
28
+ torch.manual_seed(0) # Set for reproducibility
29
+ for name, param in model.named_parameters():
30
+ param.data = torch.randn_like(param)
31
+ print(model.num_parameters()) # 4122784
32
+
33
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
34
+ ```
35
 
36
 
37
  ## Model Details