mlx-community
/

Falcon3-Mamba-7B-Instruct-4bits

Text Generation

4-bit precision

Model card Files Files and versions

Goekdeniz-Guelmez commited on Feb 14

Commit

8bda826

·

verified ·

1 Parent(s): 766ebbf

Update config.json

Files changed (1) hide show

config.json +1 -0

config.json CHANGED Viewed

@@ -11,6 +11,7 @@
     "initializer_range": 0.1,
     "intermediate_size": 8192,
     "layer_norm_epsilon": 1e-05,
     "mixer_rms_eps": 1e-05,
     "model_type": "falcon_mamba",
     "num_hidden_layers": 64,

     "initializer_range": 0.1,
     "intermediate_size": 8192,
     "layer_norm_epsilon": 1e-05,
+    "max_position_embeddings": 32768,
     "mixer_rms_eps": 1e-05,
     "model_type": "falcon_mamba",
     "num_hidden_layers": 64,