Hugging Face's logo

nithiyn
/
mathstral-neuron

mistral
Model card Files Community

Mathstral compiled for Neuron It has been compiled to run on an inf2.24xlarge instance on AWS. Note that while the inf2.24xlarge has 12 cores, this compilation uses 12.

SEQUENCE_LENGTH = 4096

BATCH_SIZE = 4

NUM_CORES = 12

PRECISION = "bf16"

Downloads last month
5
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Collection including nithiyn/mathstral-neuron