Hugging Face's logo

tachyphylaxis
/
lumikabra-195b_v0.4

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
tachyphylaxis commited on
Commit
e091eee
·
verified ·
1 Parent(s): 1e66da8

Add files using upload-large-folder tool

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +39 -0
  2. config.json +27 -0
  3. mergekit_config.yml +9 -0
  4. model-00001-of-00081.safetensors +3 -0
  5. model-00002-of-00081.safetensors +3 -0
  6. model-00003-of-00081.safetensors +3 -0
  7. model-00004-of-00081.safetensors +3 -0
  8. model-00005-of-00081.safetensors +3 -0
  9. model-00006-of-00081.safetensors +3 -0
  10. model-00007-of-00081.safetensors +3 -0
  11. model-00008-of-00081.safetensors +3 -0
  12. model-00009-of-00081.safetensors +3 -0
  13. model-00010-of-00081.safetensors +3 -0
  14. model-00011-of-00081.safetensors +3 -0
  15. model-00012-of-00081.safetensors +3 -0
  16. model-00013-of-00081.safetensors +3 -0
  17. model-00014-of-00081.safetensors +3 -0
  18. model-00015-of-00081.safetensors +3 -0
  19. model-00016-of-00081.safetensors +3 -0
  20. model-00017-of-00081.safetensors +3 -0
  21. model-00018-of-00081.safetensors +3 -0
  22. model-00019-of-00081.safetensors +3 -0
  23. model-00020-of-00081.safetensors +3 -0
  24. model-00021-of-00081.safetensors +3 -0
  25. model-00022-of-00081.safetensors +3 -0
  26. model-00023-of-00081.safetensors +3 -0
  27. model-00024-of-00081.safetensors +3 -0
  28. model-00025-of-00081.safetensors +3 -0
  29. model-00026-of-00081.safetensors +3 -0
  30. model-00027-of-00081.safetensors +3 -0
  31. model-00028-of-00081.safetensors +3 -0
  32. model-00029-of-00081.safetensors +3 -0
  33. model-00030-of-00081.safetensors +3 -0
  34. model-00031-of-00081.safetensors +3 -0
  35. model-00032-of-00081.safetensors +3 -0
  36. model-00033-of-00081.safetensors +3 -0
  37. model-00034-of-00081.safetensors +3 -0
  38. model-00035-of-00081.safetensors +3 -0
  39. model-00036-of-00081.safetensors +3 -0
  40. model-00037-of-00081.safetensors +3 -0
  41. model-00038-of-00081.safetensors +3 -0
  42. model-00039-of-00081.safetensors +3 -0
  43. model-00040-of-00081.safetensors +3 -0
  44. model-00041-of-00081.safetensors +3 -0
  45. model-00042-of-00081.safetensors +3 -0
  46. model-00043-of-00081.safetensors +3 -0
  47. model-00044-of-00081.safetensors +3 -0
  48. model-00045-of-00081.safetensors +3 -0
  49. model-00046-of-00081.safetensors +3 -0
  50. model-00047-of-00081.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - schnapper79/lumikabra-123B_v0.4
4
+ library_name: transformers
5
+ tags:
6
+ - mergekit
7
+ - merge
8
+
9
+ ---
10
+ # lumikabra-195b_v0.4
11
+
12
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
+
14
+ ## Merge Details
15
+ ### Merge Method
16
+
17
+ This model was merged using the passthrough merge method.
18
+
19
+ ### Models Merged
20
+
21
+ The following models were included in the merge:
22
+ * [schnapper79/lumikabra-123B_v0.4](https://huggingface.co/schnapper79/lumikabra-123B_v0.4)
23
+
24
+ ### Configuration
25
+
26
+ The following YAML configuration was used to produce this model:
27
+
28
+ ```yaml
29
+ dtype: bfloat16
30
+ merge_method: passthrough
31
+ slices:
32
+ - sources:
33
+ - layer_range: [0, 70]
34
+ model: schnapper79/lumikabra-123B_v0.4
35
+ - sources:
36
+ - layer_range: [18, 88]
37
+ model: schnapper79/lumikabra-123B_v0.4
38
+
39
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "schnapper79/lumikabra-123B_v0.4",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 140,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.45.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ dtype: bfloat16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [0, 70]
6
+ model: schnapper79/lumikabra-123B_v0.4
7
+ - sources:
8
+ - layer_range: [18, 88]
9
+ model: schnapper79/lumikabra-123B_v0.4
model-00001-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8cb8d2d4e1a7b6387c82d8a69a4ec9d65cbcb047abb7adda5733475b224cbfe
3
+ size 4378928504
model-00002-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:395ce11f2c53b40fcb7c04f73afefd0067932dbbd958ce991da0be71ea35842d
3
+ size 4907411088
model-00003-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1850c7ae40dd56ac29d4b0bc99939515f98ccf25b483dfccca6c1527cc12ef5
3
+ size 4806747904
model-00004-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ea89bf874b1aac34613a38f7fd625de6ced4dedd8509bd7e834b230ed27a856
3
+ size 4831938544
model-00005-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a42d8d3b7406fd50c4c6db7bd8aa242837a1a5119ab10935467a21a3d405560c
3
+ size 4831938552
model-00006-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5736e185eaceab2dc8439bfd62984183a030125883a5b96f77e0f0015deca627
3
+ size 4907411096
model-00007-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:848205db3ff0ca142cc3ecd8382168f5424dc6ea27d7f1bd643703aeb1f23a4e
3
+ size 4907435776
model-00008-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af8711fbaf2c7b6b44bf5c690e0790ef0657356754912de9c275c342026aa40c
3
+ size 4781557264
model-00009-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30406cbb9ba4537cf69f00436f74a363a6338fd9415ca0c19fb3ab3fda55d670
3
+ size 4781656640
model-00010-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29aea28819cac428a11a9adba1a5f595eb0ec04ee0231005e20ec675e33e9740
3
+ size 4882220464
model-00011-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7617af997d11791367776b7bd05d907a8a43d119925592c355060974d1c251f2
3
+ size 4932601720
model-00012-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04bd2cb4422f664adb3c87acbaa13d59d6166da833c1c78b8b1451ad9403539e
3
+ size 4781557264
model-00013-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b04715cbeddbe3143d259c50db1e665dae564e2c57f6f3039de5feccaa3d07eb
3
+ size 4831938552
model-00014-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5eeddd84e513886295853499e2a30800f6ce7cb1070a38ea0661adf6c61b5b46
3
+ size 4831938544
model-00015-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab485b52a05a8d0e094b9b270cea60ac758408d62f3a7d13ed1e91cb662b1644
3
+ size 4831938552
model-00016-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b4669b838438d81bc71d74dc1416a58d9d188f9929bda2df183110b6e4bedb5
3
+ size 4831938552
model-00017-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f98682664a540a33d50d8bf0110643f893e501d26e0f1fcf471500358237f1c
3
+ size 4831938552
model-00018-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f39906a3f77d62b3435b919a90b402307fd74427d2a2b87201677d1bc2f61d0f
3
+ size 4882220464
model-00019-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e86c4b2e109949932f7c4f6ead0d5fc764e801caced66c9ebb80c085d9e5a2b3
3
+ size 4932601720
model-00020-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:994a68424ef3a3d3aa771818dfce5c902f853427490607623c250faf066f95be
3
+ size 4781557264
model-00021-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a7355d13206260e171e49fcbf21db1cac6c54fdf658e4cd14c8e836c1a0cbcf
3
+ size 4781656640
model-00022-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d231d53db8bd54f791ea6a3c3498cada05af898b7541d7c29f76a4a8a21fb13e
3
+ size 4882220464
model-00023-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c4b1bf25522ca133f59193e6516976a62bf918382d5aad9a33fe7d11559c5d7
3
+ size 4932601720
model-00024-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2917aaddb9b095ced536af7f6fafe5fad57fb713c77559b8f879e3b49669584
3
+ size 4781557264
model-00025-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3b5a623875016fbaf911a73645d5cb1c89e56c23e22c9f11e9b4826e8c6a7ae
3
+ size 4831938552
model-00026-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b40c3f1b420dbfdb6636bf0537b43033ce7ce4f349febb8648ff1a8dab2d5af
3
+ size 4831938544
model-00027-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22783a0abaeff0d0d7fb9638ca64702acc72234c3cca7ad56c2edbd473539b8d
3
+ size 4831938552
model-00028-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38d7cc90280d7ac3be85ecf72a7fb313fe44d7ca4a648569283fbefb3d9f7815
3
+ size 4831938552
model-00029-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cf662291bf430bf0ca1f57556838f37172c9e012a7fb044467b07657343aa27
3
+ size 4831938552
model-00030-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c00ed3483981e193eb43c9bc9e1b48b6a6dd01a58068a23e717b74dc6bbf3464
3
+ size 4882220464
model-00031-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:921ddda7b60760890fa654a5aed1e8737a3f182599619882187dd576ffa36029
3
+ size 4932601720
model-00032-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b96a2e513c408e26966dc975a00e02e6b31d3303522cded1586744702c440528
3
+ size 4781557264
model-00033-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc25adbd6aabf4d52afed05356fb0167815938a6df669498fe83411eb886d92c
3
+ size 4781656640
model-00034-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5558c53ce720fc5d5a47ec78e0392d7a614af42ea541b472a89c6a4556aadb45
3
+ size 4882220464
model-00035-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a765aa7dc62c26e4e87d8bb3cc95d3f0382556c3d203f2ae73a42a388f06a10d
3
+ size 4932601720
model-00036-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5228c7d435a185b06bd2c85e2d2891761bac831e79f7fc477e5295cafc6ce22f
3
+ size 4781557264
model-00037-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd09b2db1f94a2fdccd3771c56ee8f5ecc27f21ed760ac27e9ce81e48ae738b2
3
+ size 4831938552
model-00038-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:59a5ae024a83cee31a02aefad52a9913365654511c665348c29e6feca0ffaaf6
3
+ size 4831938544
model-00039-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9101b4b068d43e0dfabcece808417ca4a1fe7c5f075c346e9adff748abfb0ed
3
+ size 4831938552
model-00040-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af4bc13dcf3c6b7e500fbb9b5b8f325d63690a7dc444b2943e6eb1a2fd674764
3
+ size 4831938552
model-00041-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a67a22cc0c2e2ac0f876d324114bfd63b910aba86d627cbeb495fda30dbaafc8
3
+ size 4831938552
model-00042-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4ebe245b3ef4754ec3f4fcba4e4cb404eb0d2b963b65dcb9cc6130b0cfab202
3
+ size 4882220464
model-00043-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57bab80ccda1ee64f152be64c444acbb24174abb1ee59459821802631c900dd1
3
+ size 4932601720
model-00044-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a33d7f8671234f781edb446aeb0021bff8b837fb7bfd8cd932bd2a0277df302e
3
+ size 4781557272
model-00045-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62fa9aa9f6d00f2cf418a4df2716de15b5493adbc1db13124f20e16f59f019c5
3
+ size 4781656632
model-00046-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e9fdbba7370036dea47e0ae53de292562b9f19aa541654ff58ee5aa753f5551
3
+ size 4882220464
model-00047-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e3fd59da322d2ad70d27cbf22688255e8c1e91404cabef32240d0565b73c446
3
+ size 4932601728