Add model files

README.md

@@ -1,3 +1,77 @@
 ---
+base_model: mixedbread-ai/mxbai-embed-2d-large-v1
+language:
+- en
+library_name: model2vec
 license: mit
+model_name: red-beetle-base-v1.1
+tags:
+- embeddings
+- static-embeddings
+- sentence-transformers
 ---
+# 🪲 red-beetle-base-v1.1 Model Card
+
+<div align="center">
+  <img width="75%" alt="Beetle logo" src="./assets/beetle_logo.png">
+</div>
+
+> [!TIP]
+> Beetles are some of the most diverse and interesting creatures on Earth. They are found in every environment, from the deepest oceans to the highest mountains. They are also known for their ability to adapt to a wide range of habitats and lifestyles. They are small, fast and powerful!
+
+The beetle series of models are made as good starting points for Static Embedding training (via TokenLearn or Fine-tuning), as well as decent Static Embedding models. Each beetle model is made to be an improvement over the original **M2V_base_output** model in some way, and that's the threshold we set for each model (except the brown beetle series, which is the original model).
+
+This model has been distilled from `mixedbread-ai/mxbai-embed-2d-large-v1`, with PCA at 1024 dimensions, Zipf and SIF re-weighting, learnt from a subset of the FineWeb-Edu sample-10BT dataset. It outperforms the original M2V_base_output model in all tasks.
+
+## Version Information
+
+- **red-beetle-base-v0**: The original model, without using PCA or Zipf. The lack of PCA and Zipf also makes this a decent model for further training.
+- **red-beetle-base-v1**: The original model, with PCA at 1024 dimensions and (Zipf)^3 re-weighting.
+- **red-beetle-small-v1**: A smaller version of the original model, with PCA at 384 dimensions and (Zipf)^3 re-weighting.
+- **red-beetle-base-v1.1**: The original model, with PCA at 1024 dimensions, Zipf and SIF re-weighting, learnt from a subset of the FineWeb-Edu sample-10BT dataset.
+- **red-beetle-small-v1.1**: A smaller version of the original model, with PCA at 384 dimensions, Zipf and SIF re-weighting, learnt from a subset of the FineWeb-Edu sample-10BT dataset.
+
+## Installation
+
+Install model2vec using pip:
+
+```bash
+pip install model2vec
+```
+
+## Usage
+
+Load this model using the `from_pretrained` method:
+
+```python
+from model2vec import StaticModel
+
+# Load a pretrained Model2Vec model
+model = StaticModel.from_pretrained("bhavnicksm/red-beetle-base-v1.1")
+
+# Compute text embeddings
+embeddings = model.encode(["Example sentence"])
+```
+
+Read more about the Model2Vec library [here](https://github.com/MinishLab/model2vec).
+
+## Comparison with other models
+
+Coming soon...
+
+## Acknowledgements
+
+This model is made using the [Model2Vec](https://github.com/MinishLab/model2vec) library. Credit goes to the [Minish Lab](https://github.com/MinishLab) team for developing this library.
+
+## Citation
+
+Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
+
+```bibtex
+@software{minishlab2024model2vec,
+  authors = {Stephan Tulkens, Thomas van Dongen},
+  title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
+  year = {2024},
+  url = {https://github.com/MinishLab/model2vec},
+}
+```

config.json

@@ -0,0 +1,12 @@
+{
+    "model_type": "model2vec",
+    "architectures": [
+        "StaticModel"
+    ],
+    "tokenizer_name": "mixedbread-ai/mxbai-embed-2d-large-v1",
+    "apply_pca": null,
+    "apply_zipf": false,
+    "hidden_dim": 1024,
+    "seq_length": 1000000,
+    "normalize": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:762c14708e0040223c5cbb4d7b8f474166a0fabf4fa5d7fbd82d3007902969ca
+size 120946776

modules.json

@@ -0,0 +1,8 @@
+[
+    {
+        "idx": 0,
+        "name": "0",
+        "path": ".",
+        "type": "sentence_transformers.models.StaticEmbedding"
+    }
+]