Create README.md

README.md

@@ -0,0 +1,51 @@
+---
+language:
+- en
+pipeline_tag: text-generation
+inference: false
+tags:
+- mistral
+- inferentia2
+- neuron
+- neuronx
+---
+# Neuronx for [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) - Updated Mistral 7B Model on [AWS Inferentia2](https://aws.amazon.com/ec2/instance-types/inf2/) 
+
+This model has been exported to the `neuron` format using specific `input_shapes` and `compiler` parameters detailed in the paragraphs below.
+
+Please refer to the 🤗 `optimum-neuron` [documentation](https://huggingface.co/docs/optimum-neuron/main/en/guides/models#configuring-the-export-of-a-generative-model) for an explanation of these parameters.
+
+Note: To compile the mistralai/Mistral-7B-Instruct-v0.2 on Inf2, you need to update the model config sliding_window (either file or model variable) from null to default 4096.
+
+## Usage with 🤗 `optimum-neuron`
+
+```python
+>>> from optimum.neuron import pipeline
+
+>>> p = pipeline('text-generation', 'davidshtian/Mistral-7B-Instruct-v0.2-neuron-1x2048-2-cores')
+>>> p("My favorite place on earth is", max_new_tokens=256, do_sample=True, top_k=50)
+[{'generated_text': 'My favorite place on earth is the ocean. It is where I feel most
+at peace. I love to travel and see new places. I have a'}]
+```
+
+This repository contains tags specific to versions of `neuronx`. When using with 🤗 `optimum-neuron`, use the repo revision specific to the version of `neuronx` you are using, to load the right serialized checkpoints.
+
+## Arguments passed during export
+
+**input_shapes**
+
+```json
+{
+  "batch_size": 1,
+  "sequence_length": 2048,
+}
+```
+
+**compiler_args**
+
+```json
+{
+  "auto_cast_type": "bf16",
+  "num_cores": 2,
+}
+```